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Skill  Training  Using  Adaptive  Technology: 

A  Better  Way  To  Hover 

INTRODUCTION 

The  goal  of  this  Phase  1  Small  Business  Innovation  Research  (SBIR)  program  was  to  determine 
the  feasibility  of  developing  and  implementing  an  automated,  adaptive  hover  training  controller 
based  on  human  performance  models  and  novel  feedback  techniques  for  Student  Pilots  (SPs)  in 
Initial  Entry  Rotary  Wing  (IERW)  training.  The  target  system  for  the  controller  is  the  Intelligent 
Flight  Trainer  (IFT),  a  hover  training  research  device,  operated  by  the  U.S.  Army  Research 
Institute,  Ft.  Rucker  Field  Office,  Ft.  Rucker,  Alabama  (ARI).  This  work  was  conducted 
between  April  and  August  2000  by  Monterey  Technologies,  Inc.  (MTI)  with  significant 
consulting  from  Dr.  Anthony  P.  Ciavarelli  of  the  U.S.  Naval  Postgraduate  School,  Monterey, 
California  and  Mr.  Charles  Asbury  of  ThoughtWave,  LLC,  Torrance,  California. 

Aviation  training  programs  are  some  of  the  longest,  most  costly,  and  demanding  in  Army.  ARI 
is  continually  conducting  research  to  find  ways  to  improve  the  efficiency  and  reduce  the  cost  of 
training  SPs  to  become  Army  Aviators.  The  ability  to  hover  a  helicopter  is  one  of  the 
fundamental  skills  required  early  in  the  IERW  program.  At  present,  IERW  is  conducted  on  a 
training  time  basis.  That  is,  within  a  small  range  of  variation,  all  students  receive  the  same 
number  of  hours  of  training  in  lock-step  fashion.  A  more  efficient  training  program  would  be 
performance  based.  SPs  would  advance  through  the  program  based  on  demonstrated  individual 
proficiency.  While  Instructor  Pilots  (IPs)  have  the  responsibility  to  assess  performance,  it  would 
highly  desirable  to  have  automated  systems  that  would  adapt  the  training  challenge  within  a 
lesson  to  the  current  level  of  the  SP’s  performance  and  also  assess  when  the  SP  is  capable  of 
advancement  to  a  new  lesson  or,  in  some  cases,  be  regressed  to  a  previous  lesson. 

ARMY  AVIATION  TRAINING 

The  primary  phase  of  the  US  Army  IERW  flight  training  syllabus  for  helicopter  pilots  requires 
approximately  10  weeks  to  complete.  During  this  time  each  student  receives  about  60  hours  of 
instruction  in  the  TH-67  Creek  helicopter.  Subsequent  combat  skills  training  requires  8  weeks 
with  4  additional  weeks  for  Night  Vision  Goggle  (NVG)  training,  and  uses  49  hours  in  the 
aircraft.  So,  a  pilot  making  normal  progress  will  have  received  at  least  28  weeks  of  training  and 
used  about  129  hours  of  aircraft  time  even  before  beginning  training  in  advanced  aircraft  such  as 
the  UH-60  or  AH-64  (Wightman,  1999).  All  of  this  training  is  conducted  at  Fort  Rucker,  AL  in 
TH-67  Creek  helicopters  (see  Figure  1).  The  TH-67  is  a  variant  of  the  Bell  Jet  Ranger 
helicopter.  Simulators  are  used  only  during  non-contact  (i.e.,  instrument)  training,  which  follows 
IERW. 
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Figure  1.  TH-67  “Creek”  helicopter  used  in  Initial  Entry  Rotary'  Wing  Flight  Training. 


IERW  flight  training  consists  of  59  flight  hours  flown  oxer  49  flight  training  days  (US  Army, 
1999),  Simulators  are  not  used  during  IERW  training  at  the  present  time.  One  of  the  first 
challenges  for  the  SP  is  to  learn  to  hover  the  helicopter.  Most  SPs  require  6  to  8  hours  to  hover 
competently.  By  about  10  hours  of  flight  time  they  should  be  able  to  maneuver  the  aircraft  from 
one  position  to  another  in  a  hover  taxi.  Most  students  solo  with  about  22  hours  in  the  aircraft. 

ADAPTIVE  TRAINING 

Adaptive  training  is  the  process  of  altering  the  del  i\ cry  of  instruction  to  match  the  student’s 
current  skill  level  and  cognitive  model  of  the  task.  As  there  are  individual  differences  in  the 
rates  students  learn  complex  motor  skills  and  the  style  of  instruction  that  is  most  effective, 
adaptive  training  systems  must  tailor  the  training  to  an  individual  student. 

The  ARI IFT  is  an  example  of  a  device  employing  adaptive  training  algorithms.  The  stability  of 
the  IFT  is  varied  based  on  the  measured  performance  of  the  pilot  within  a  simulated  flight 
session.  When  the  pilot  is  performing  poorly  ( i.e..  position  and  altitude  variability  is  high  while 
attempting  to  hover)  the  aircraft  model  is  made  more  stable.  As  the  pilot  maintains  the 
parameters  more  closely,  the  stability  of  the  simulated  helicopter  is  decreased  until,  at  the  lowest 
level  of  stability,  the  simulator  flies  like  the  actual  helicopter.  If  the  pilot's  performance 
deteriorates,  then  the  stability  of  the  simulated  aircraft  is  increased. 

The  IFT  also  provides  automated  verbal  feedback  to  the  pilot.  For  example,  if  the  aircraft  is 
drifting  outside  of  predetermined  limits,  the  voice  system  w  ill  alter  the  pilot  to  the  drift.  If  an 
appropriate  correction  is  not  made  the  voice  system  will  then  alert  the  pilot  to  the  problem  and 
recommend  a  control  input  to  correct  the  problem. 
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As  currently  configured,  the  IFT  does  not  actually  use  a  model  of  pilot  of  the  instructor  or  of  the 
student  in  determining  how  and  when  to  alter  the  simulator  or  offer  verbal  feedback  to  the  pilot. 
Changes  in  stability  and  verbal  feedback  are  based  solely  on  the  variability  of  the  aircraft’s 
position  over  time. 

Adaptive  training  is  expected  to  be  more  efficient  than  in  traditional  lockstep  programs  because 
the  student  is  always  performing  an  appropriately  difficult  psychomotor  task  As  the  skill  of  the 
individual  student  increases,  the  task  is  made  more  difficult.  In  turn,  as  the  student’s  skills 
improve  enough  to  allow  adequate  performance  at  the  new  difficulty  level,  the  difficulty  is  again 
incremented.  Should  the  student’s  performance  decline,  then  the  stability  of  the  simulated 
helicopter  is  increased.  The  student  does  not  spend  any  time  performing  tasks  at  a  level  already 
mastered,  nor  is  the  student’s  task  so  difficult  that  it  interferes  with  the  improvement  of  the 
desired  skills.  In  many  ways,  the  adaptive  system  mimics  the  performance  of  an  expert 
instructor. 

PROGRAM  OBJECTIVE 

The  goal  of  this  program  is  to  determine  the  feasibility  of  designing  and  implementing  a 
successful  adaptive  training  program  to  instruct  SPs  the  skills  and  knowledge  needed  to  hover. 

INSTRUCTIONAL  CONCEPT 

Figure  2  is  a  block  diagram  showing  the  key  elements  of  MTI’s  conception  of  adaptive  training 
system  for  a  hover  trainer. 

Figure  2.  Block  Diagram  of  an  Adaptive  Training  System. 
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In  this  figure,  the  “IP  Expert  System”  receives  performance  measures  from  the  SP  flying  the 
training  simulator.  It  also  receives  input  on  the  previous  performance,  performance  trends,  and 
effects  of  previous  trainer  adaptations  for  this  particular  student.  This  expert  system  uses  the 
historical  data  and  the  current  performance  measures  to  assess  the  state  of  the  student.  If  the 
student  has  not  mastered  the  task  or,  more  accurately,  is  not  performing  at  the  appropriate  level 
for  that  point  in  the  training  schedule,  then  the  system  diagnoses  the  student’s  difficulty.  If  the 
student  demonstrates  mastery  of  the  task,  then  the  system  moves  on  to  the  next  training  objective 
and  does  not  waste  time  on  the  already  mastered  task. 

STUDENT  PERFORAMCE  HISTORY 

The  student  performance  history  information  is  critical  to  this  process  for  several  reasons.  First, 
and  most  obviously,  the  expectations  and  performance  criteria  for  a  SP  on  the  first  flight  are 
different  than  for  a  SP  who  has  already  had  several  hours  of  practice.  Similarly,  the  type  of 
adaptation  and  feedback  that  is  appropriate  will  depend  on  the  state  of  the  SP.  When  just 
beginning  it  is  expected  that  the  SP  will  be  attempting  to  understand  the  plant  dynamics.  That  is, 
the  relationships  and  interactions  between  the  controls  and  changes  in  attitude  and  position  of  the 
helicopter.  The  SP  will  be  under  high  cognitive  workload  to  identify  these  relationships  and  it 
will  be  expected  that  the  types  of  adaptation  and  feedback  will  be  focused  on  making  the  plant 
dynamics  apparent.  Visual  cue  augmentation  and  slower  than  real  time  simulation,  along  with 
alteration  of  the  helicopter’s  stability,  are  candidate  methods  of  adaptation  at  this  phase  of 
learning.  Later,  after  a  some  practice  which  allows  the  SP  to  understand  how  the  control  work 
and  interact,  the  focus  of  training  will  normally  shift  to  developing  the  fine  psychomotor  skills 
needed  to  control  the  aircraft  in  real  time  without  “thrashing”  or  “under-controlling”  the  controls. 
At  this  point  in  the  skill  acquisition  process  altering  the  proprioceptive  feedback  by  either 
damping  the  control  motion  velocity  to  an  appropriate  level  (or  removing  damping  if  the  SP 
tends  to  feed  in  controls  too  slowly),  or  perhaps  limiting  the  control  authority,  might  be  more 
effective  ways  to  adapt  the  trainer. 

A  second,  but  perhaps  less  obvious,  use  of  the  historical  data  is  to  predict  future  performance  of 
the  SP.  The  adaptive  system  should  have  the  capability  to  categorize  the  student.  Where 
students  who  fit  the  same  pattern  have  previously  encountered  difficulties  at  later  stages  of 
training,  the  trainer  should  alter  the  training  in  an  anticipatory  fashion.  By  adapting  the  trainer 
early,  it  may  be  possible  for  the  current  student  to  avoid  the  problems  encountered  by  students 
how  have  passed  before.  As  an  example,  consider  the  following  hypothetical  pattern  of  SP 
performance.  At  an  early  point  in  training,  students  tend  to  fall  into  two  categories;  those  who 
over-control  the  fore-aft  cyclic  and  those  who  tend  to  under-control  it.  Assume  that  students 
who  over-control  the  fore-aft  cyclic  later  tend  focus  on  this  so  much  that  they  fail  to  use 
sufficient  lateral  cyclic.  In  this  example,  the  IP  Expert  System  would  diagnose  the  current 
problem  and  predict  the  development  of  the  future  problem.  This  would  allow  the  IT  system  to 
modify  the  trainer  to  discourage  thrashing  the  stick  in  the  fore-aft  direction  without  letting  active 
lateral  stick  control  become  extinguished. 

When  a  task  is  considered  “mastered”  that  it  does  not  necessarily  mean  that  the  SP  is  performing 
at  the  level  of  an  experienced,  expert  helicopter  pilot.  In  this  context  it  means  that  the  student 
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has  reached  the  level  of  proficiency  required  to  begin  learning  new  training  objectives.  This  is 
no  different  than  the  current  practice  in  IERW  training.  Once  a  student  can  perform  a  maneuver 
“good  enough”  an  IP  moves  on  to  the  next  challenge. 

THE  INSTRUCTIONAL  TECHNOLOGIST  EXPERT  SYSTEM 

The  Instructional  Technologist  (IT)  Expert  System  receives  the  diagnosis  from  the  IP  system  as 
input.  The  IT  system  using  a  prescriptive  process  to  determine  the  type  and  magnitude  of  the 
change  in  the  simulator,  if  any,  that  is  appropriate.  Once  the  change  is  determined,  this  system 
causes  the  change  to  be  implemented  in  the  trainer.  In  those  instances  where  the  training 
prescription  is  to  provide  voice  feedback  to  the  student,  the  IT  system  causes  a  the  message  to  be 
broadcast  to  the  SP. 

When  the  objectives  for  that  session  have  been  reached  then  training  is  considered  complete  for 
that  session.  Other  criteria  may  also  be  used  to  end  a  session.  For  example,  if  the  student  is  not 
progressing  and  becoming  frustrated  to  the  point  that  training  is  impaired  in  that  session,  then  the 
system  would  terminate  the  session  and  store  information  that  would  allow  it  to  resume  training 
where  it  left  off.  This  would  provide  the  student  an  opportunity  to  “settle  down”  and  think 
though  the  problem. 

LITERATURE  REVIEW 

A  review  of  the  literature  review  relevant  to  learning  and  adaptive  training  was  conducted  as  a 
part  of  this  effort.  This  review  showed  that  the  instructional  prescriptions  for  complex  skill 
training,  unlike  prescriptions  for  cognitive  training,  are  not  well  developed  or  validated 
operationally.  Therefore,  at  this  time  one  can  only  extrapolate  from  the  results  of  laboratory 
studies. 

The  literature  review  is  contained  in  its  entirety  in  Appendix  1 .  Key  points  from  the  literature 
review  are  synopsized  below. 

CUE  HIGHLIGHTING 

Gibson  (1966)  and  others  (Merrill,  1972)  have  suggested  that  skill  training  can  be  facilitated  by 
highlighting  important  cues  for  the  student.  In  other  words,  the  instructional  system  should  be 
designed  so  that  the  student’s  attention  is  focused  on  the  visual,  vestibular,  auditory,  and 
proprioceptive  cues  that  indicate  changes  in  helicopter  state.  As  an  example,  one  might  highlight 
the  tip  path  plane  in  the  visual  scene  so  that  the  SP  begins  to  attend  to  it,  rather  than  waiting  for 
the  SP  to  discover  the  relationship  between  changes  in  the  tip  path  and  future  changes  in  the 
helicopter’s  position  or  velocity. 

MODELING 

Gibson  also  suggests  that  demonstrating  the  relationship  between  the  cues  and  the  actions  that 
the  student  should  make  in  response  to  changes  in  those  cues  will  improve  training.  Modeling 
the  correct  control  motions  is  certainly  important  in  terms  of  giving  the  SPs  an  example  of  the 
performance  that  should  be  ultimately  reached.  However,  as  it  is  unlikely  that  SPs  thrashing  the 
controls  don’t  realize  that  they  are  over  controlling  the  aircraft.  It  is  likely  that  modeling  at  a 
more  fundamental  level  would  benefit  a  SP.  In  particular,  what  may  be  required  in  addition  to 
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modeling  the  ultimate  performance  is  to  use  the  modeling  effort  to  provide  the  student  an 
understanding  of  the  “plant  dynamics.”  That  is,  the  instructional  system  should  use  modeling  to 
augment  the  cognitive  training  on  the  effects  of  each  control  and  the  interactive  effects  of  the 
controls  on  the  aircraft. 

TASK  DECOMPOSITION 

One  training  strategy  is  to  decompose  the  task  into  segments  and  allow  the  student  to  practice 
each  of  the  segments  individually  before  performing  the  task  in  its  entirety.  It  appears  from  the 
literature  that  this  approach  can  be  successful  under  certain  conditions.  Wightman  and  Lintern 
(1985)  found  that  segmentation,  particularly  when  the  task  is  trained  using  backward  chaining, 
can  be  effective.  They  recommend: 

•  Isolating  consistent  task  elements  for  focused  practice 

•  Providing  repeated  practice  with  feedback 

•  Practicing  critical  tasks  separately  to  avoid  information  overload 

•  Training  to  an  acceptable  level  of  accuracy 

•  Incorporating  extrinsic  motivational  conditions 

Backward  chaining  has  been  demonstrated  to  be  an  effective  training  approach  in  teaching  the 
skills  required  for  air-to-ground  bombing  (Bailey,  Hughes,  and  Jones,  1980)  and  in  teaching 
carrier  landing  (Wightman,  1983). 

PROGRESSIVE  DIFFICULTY 

There  results  reported  in  the  literature  on  the  effectiveness  of  changing  the  difficulty  of  a  task 
suggest  that  this  is  not  a  particularly  effective  approach  to  training.  Wightman  and  Sistrunk 
(1987)  found  that  the  progressive  difficulty  approach  was  inferior  in  terms  of  training 
effectiveness  when  compared  to  part  task  segmentation.  Mane  investigated  the  effect  of  altering 
the  speed  of  a  video  game  on  skill  acquisition.  In  this  study  the  game  was  slowed  down  for 
beginners  and  increased  the  speed  as  skills  increased.  In  other  conditions,  the  students  either 
practiced  the  game  at  normal  speed  or  received  part  task  training.  The  learning  rates  of  students 
in  the  progressive  difficulty  and  normal  speed  conditions  were  inferior  to  that  of  students  who 
received  part  task  training. 

While  the  available  data  does  not  rule  out  the  possibility  that  selecting  other  rates  of  changing 
difficulty  will  be  effective,  it  appears  that  this  approach  is  not  highly  likely  to  be  particularly 
successful  in  the  hover  training  application. 

COGNITIVE  TRAINING 

Smith  (1984)  examined  the  effects  of  providing  cognitive  training  before  or  during  skill 
acquisition  training.  There  were  four  groups  in  this  study.  One  group  received  cognitive 
training  prior  to  the  start  of  skill  acquisition  training.  A  second  group  received  the  same  pre-skill 
acquisition  training  as  the  first  group  coupled  with  sporadic  cognitive  training  during  the  skill 
acquisition  training.  A  third  group  received  no  cognitive  training  prior  to  beginning  skill 
acquisition  training,  but  received  sporadic  cognitive  training  concurrently  with  skill  acquisition 
training.  A  control  group  received  no  cognitive  training  either  before  or  during  skill  acquisition 
training.  Interestingly,  Smith  also  found  that  cognitive  training  had  a  beneficial  effect  only  when 
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presented  before  beginning  kill  acquisition  training;  providing  cognitive  training  sporadically 
during  skill  acquisition  had  an  adverse  effect.  The  conclusion  reached  is  that  the  timing  of 
cognitive  training  relative  to  skill  acquisition  training  is  critical. 

The  implication  of  this  finding  is  that  IERW  should  be  structured  so  that  the  ground  school 
sessions  covering  fundamentals  should  precede  the  start  of  flight  training.  Further,  these  results 
suggest  that  cognitive  training  for  specific  maneuver  should  always  precede  the  flights  in  which 
the  maneuver  is  practiced. 

OBSERVATIONAL  LEARNING 

The  literature  generally  shows  that  observational  learning  is  more  effective  for  cognitive  tasks 
than  for  tasks  that  are  predominately  psychomotor  tasks  (Goettl  and  Gomez,  1 995).  This  result 
is  consistent  with  the  notion  that  improvements  in  psychomotor  performance  is  marked  by 
increased  automatization  of  the  task.  Simply,  observation  does  not  provide  the  student  with 
adequate  opportunities  to  allow  the  skill  to  become  “ingrained”. 

EVALUATION  OF  ARTIFICIAL  INTELLIGENCE  (AI)  TOOLS 

An  intelligent,  adaptive  training  system  will  require  software  be  created  that  can  (1)  assess  the 
student  and  identify  training  needs  (i.e.,  a  model  of  an  expert  IP)  and  (2)  can  translate  the 
identified  training  needs  into  changes  in  the  training  delivery  system  (i.e.,  a  model  of  an 
instructional  technology  expert).  It  would  be  more  efficient  and  present  less  technical  risk  to  the 
program  to  create  these  models  using  COTS  tools  developed  to  support  AI  applications  than  to 
write  all  the  software  in-house. 

A  review  of  AI  approached  and  tools  was  conducted  as  a  part  of  this  program.  This  review  is 
contained  in  Appendix  2.  Key  points  from  this  review  are  presented  below. 

Initial  efforts  focused  on  identifying  the  approach  or  approaches  that  would  be  appropriate  for 
creating  these  systems.  Technologies  considered  included: 

•  Conceptual  Graph  Analysis 

•  Case  Based  Reasoning 

•  Modeling  and  Simulation 

•  Neural  Networks  and  Genetic  Algorithms 

Case  Based  Reasoning  (CBR)  is  the  approach  recommended  for  developing  the  Instructor  Pilot 
and  Instructional  Technologist  models.  The  benefits  of  CBR  that  make  this  approach  the  most 
attractive  of  those  examined  are: 

•  The  ability  to  quickly  find  solutions  to  complex  problems 

•  Discovering  decision  knowledge  using  inductive  reasoning 

•  Transferring  experience  from  skilled  specialists  to  novices 

•  Pooling  individual  experiences  into  a  shared  pool  of  domain  knowledge 

•  Demonstrating  expertise  in  domains  that  are  poorly  understood,  including  domains 
where  the  theoretical  underpinnings  of  successful  solutions  are  undeveloped. 
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•  Penetrability  -  it  is  relatively  easy  to  determine  how  the  system  reached  a  decision 

•  Maintainability  -  rules  that  do  not  support  the  objective  can  be  modified  or  deleted, 
or  new  rules  added  easily 

Once  an  approach  had  been  identified,  a  number  of  COTS  tools  were  identified  and  examined. 
The  recommendation  coming  from  this  review  is  that  the  adaptive  training  system  be  built  using 
the  Easy  Reas  oner,  CBRete++,  and  CPR  packages,  with  additional  software  to  support  the 
packages  and  integrate  the  system  being  written  in  C++. 

ADAPTIVE  TECHNIQUES 

After  the  expert  IP  has  assessed  the  student’s  performance  and  identified  a  strength  or  weakness, 
the  training  device  must  be  altered.  The  type  of  modification  made  must  be  appropriate  to  the 
particular  aspect  of  performance  to  be  modified  trough  training.  Alterations  along  a  single 
dimension,  such  as  in  terms  of  aircraft  stability  as  is  now  done  in  the  IFT,  will  be  useful  in  some 
instances,  but  are  likely  to  be  inadequate  to  address  the  entire  range  of  instructional  issues  that 
are  encountered. 

The  role  of  the  instructional  expert  model  is  to  identify  the  manner  in  which  the  simulator  should 
be  altered  to  best  suit  the  student.  Below  are  brief  descriptions  of  additional  characteristics  of 
the  training  system  that  could  be  altered  to  meet  the  needs  of  the  students.  This  list  is  intended  to 
show  the  range  of  simulator  characteristics  that  can  be  modified.  It  is  not  an  exhaustive  list.  The 
set  of  features  that  could  be  adapted  is  limited  only  by  the  creativity  of  the  people  developing  the 
system  and  the  capabilities  of  the  specific  training  simulator. 

A  major  part  of  the  process  of  developing  an  adaptive  system  will  be  measuring  the  effectiveness 
of  the  manipulations  made  to  the  training  system. 

AMBIENT  CUES 

If  the  student  is  having  trouble  judging  motion  of  the  aircraft  in  one  or  more  specific  axes,  then 
overlay  a  grid  pattern  on  the  terrain  and,  if  the  motion  is  vertical,  on  vertical  panels  around  the 
helicopter.  This  will  provide  motion  cues  that  are  more  robust  than  those  available  in  the 
relatively  sparse  visual  scenes. 

In  extreme  cases,  or  cases  where  it  is  desirable  to  increase  the  response  amplitude  of  the  pilot, 
the  motion  of  the  ambient  checkerboards  can  be  amplified.  For  example,  it  the  pilot  has  trouble 
detecting  changes  in  heading,  move  the  checker  board  through  a  larger  angle  than  that  through 
which  the  aircraft  moves. 

CONTROL  FORCE  GRADIENTS  CHANGES 

In  this  approach,  the  pilot  is  discouraged  from  making  control  movements  of  excessive 
amplitude  or  excessive  velocity  though  the  use  of  force  gradients.  For  example,  if  the  SP  moves 
the  control  too  quickly,  control  damping  is  added  so  that  whenever  the  velocity  exceeds  the 
desired  maximum  velocity  the  amount  of  force  required  is  noticeably  above  the  force  required  to 
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move  the  control  the  same  amplitude  at  a  rate  at  or  below  the  acceptable  level.  In  the  case  of 
excessively  large  amplitude  motions,  the  force  would  increase  as  the  control  reaches  and  exceeds 
the  desired  maximum. 

If,  on  the  other  hand,  the  control  motions  were  not  of  sufficient  amplitude  or  if  the  velocity  of  the 
input  was  too  slow,  then  the  force  could  be  reduced  below  the  nominal  level.  This,  I  predict, 
would  have  less  of  an  effect  than  in  the  situations  where  the  force  is  increased  when  the  limit  is 
reached. 

This  could  be  applied  to  any  control  function  where  the  operator  has  a  desired  range  of  control 
motions  and/or  amplitudes.  Driving  automobiles  is  one  example.  Student  drivers  tend  to  be 
overly  active  with  the  wheel,  as  if  they  are  modeling  the  driving  style  from  an  old,  bad  movie. 

“FLY-TO”  DISPLAYS 

Instead  of  a  predictor  display,  the  pilot  could  be  given  a  “fly  to”  display.  This  would  provide 
continuous,  instantaneous  feedback  to  the  pilot  regarding  the  direction  and  magnitude  of  the 
control  input  error.  The  pilot  would  be  able  to  use  this  display  to  develop  the  “muscle  memory” 
relating  the  aircraft  state  and  the  control  input.  The  pilot  would  need  to  be  weaned  from  this 
display  in  order  to  internalize  the  skill.  Otherwise,  the  pilot  would  be  a  cripple  and  rely  on  the 
cue  to  control  the  aircraft. 

GODS  EYE  VIEWS 

Use  a  God’s  eye  view.  In  this  approach,  the  student  pilot  (SP)  views  a  helicopter  from  a  position 
to  the  behind  the  helicopter.  The  position  might  also  be  behind  and  above  the  helicopter.  This 
will  allow  the  SP  to  learn  the  control  laws/plant  dynamics.  For  example,  the  SP  can  see  the 
effect  of  adding  lateral  cyclic  to  stop  or  start  lateral  drift.  They  will  also  be  able  to  easily  detect 
aircraft  yaw  motion  as  they  pull  in  collective. 

The  commercial  potential  includes  teaching  people  to  back  trailers.  One  could  imagine  using 
this  in  initial  truck  driver  training.  It  could  also  be  used  by  rental  agencies  (e.g.,  U-Haul)  to  let 
renters  practice  backing  a  trailer. 

Another  application  is  teaching  student  drivers  to  parallel  park  an  automobile.  In  this  case,  you 
would  probably  want  to  start  with  the  fenders  being  transparent  so  that  the  student  can  see  the 
position  of  the  tires.  Later,  the  fenders  should  be  opaque  so  that  the  student  has  to  learn  to  rely 
on  the  cues  from  just  the  position  of  the  steering  wheel  without  being  able  to  visually  ascertain 
the  position  of  the  tires. 

HIGHLIGHTING  OF  SALIENT  CUES 

There  are  cues  that  the  student  needs  to  attend  to  in  the  aircraft.  A  student  enters  the  program  not 
knowing  what  cues  to  attend  to.  These  cues  may  be  available  in  any  modality  including: 

•  Sound 

•  Visual 

•  Vestibular 
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Since  the  student  doesn’t  know  what  to  attend  to  and  what  to  ignore  (what  is  critical  vs.  what  is 
junk)  he  or  she  may  be  left  to  “discover”  the  cues  on  his  own.  This  is  inefficient  at  best,  and  may 
result  in  the  student  failing  to  recognize  and  exploit  appropriate  information  sources.  (For 
example,  students  can’t  be  expected  to  realize  the  importance  of  the  position  of  the  rotor  disc. 

As  the  disk  tips  forwards,  the  aircraft  will  pitch  and  then  move  forwards.).  The  simulator  should 
help  the  pilot  identify  and  attend  to  the  cues  needed  to  perform  maneuvers. 

MULTIPLE  MODE  ERROR  SIGNALING 

Provide  the  pilot  cues  in  a  modality  that  is  not  normally  used  in  recognizing  the  error.  For 
example,  if  the  pilot  is  working  on  aircraft  roll  control,  change  the  pitch  of  a  sound  so  that  when 
the  left  wing  is  low  the  pitch  of  the  sound  goes  up,  and  when  the  right  wing  is  low  the  pitch  of 
the  sound  goes  down.  This  could  be  used  to  cue  the  pilot  on  any  axis.  The  pilot  would  be 
“weaned”  from  this  cue  once  performance  met  some  criteria. 

PREDICTED  POSITION  DISPLAYS 

If  the  pilot  is  having  trouble  relating  the  magnitude  of  control  inputs  to  the  situation,  provide  a 
predictor  display.  The  predictor  display  would  show  the  position  of  the  aircraft  in  TBD  seconds 
given  the  current  control  input. 

You  would  want  to  experiment  with  different  orders  of  the  prediction  equation.  For  example, 
you  might  want  to  use  a  linear  extrapolation  given  the  current  velocity  of  the  aircraft  (for  a 
translational  error),  or  use  a  higher  order  equation  that  takes  the  rate  and  acceleration  into 
account  to  come  up  with  a  predicted  aircraft  position.  See  also  the  idea  of  “fly  to”  displays. 

RADIO  CHATTER  CONTROL 

Flying  is  not  done  in  a  quiet  environment.  There  are  sounds  that  students  must  learn  to  attend  to, 
and  those  that  they  must  ignore.  At  the  very  beginnings  of  flight  training,  students  can’t  (and 
probably  shouldn’t)  be  hearing  chatter  on  the  radio.  Instead,  the  system  should  eliminate  the 
radio  chatter  during  initial  flights.  Only  when  the  student  has  began  to  master  the  helicopter 
should  potential  distractions  be  added  into  the  simulation. 

As  the  student  progresses,  he  or  she  must  learn  to  use  the  radios.  Use  of  the  radios  includes 
learning  to  communicate  with  the  tower  and  possibly  other  aircraft.  Use  of  the  radios  also 
incorporates  learning  to  monitor  the  radio  for  relevant  messages  while  ignoring  those  that  are 
irrelevant.  If  the  pilot  fails  in  these  tasks,  he  or  she  will  be  either  inundated  with 
communications  to  the  point  of  being  overloaded  or  not  being  aware  of  messages  that  are 
addressed  to  that  aircraft.  Certainly,  adequate  use  of  the  radios  (listening,  responding,  and 
ignoring)  is  a  skill  that  must  be  developed  by  the  time  the  student  is  ready  to  solo. 

THREE  DIMENSIONAL  SOUND  ERROR  SIGNALING 

Use  3-D  sound  cueing  to  help  the  pilot  identify  the  direction  of  the  ideal  position  from  his 
current  position.  You  could,  in  principle,  make  a  sound  appear  to  come  from  in  front  of  or 
behind  the  pilot,  or  to  the  left  or  right  of  the  pilot,  of  from  above  or  below  the  pilot.  This  sound 
could  be  configured  so  the  pilot  flies  the  aircraft  towards  the  sound.  For  example,  if  the  aircraft 
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is  above  and  to  the  left  of  the  ideal  hover  position,  the  sound  would  be  perceived  to  be  coming 
from  a  location  below  and  to  the  right  of  the  pilot. 

The  gain  on  the  movement  of  the  sound  could  be  varied.  As  the  pilot  becomes  more  proficient, 
the  size  of  the  error  that  triggers  the  cue  could  be  made  smaller.  That  is,  at  first  the  amount  of 
error  in  aircraft  that  causes  the  cue  to  increase  is  large  (e.g.,  12  ft).  As  the  pilot  becomes  more 
proficient,  the  distance  that  the  aircraft  moves  before  the  position  of  the  sound  is  changed 
decreases  (e.g.,  is  reduced  to  3  ft).  Not  only  does  the  cue  adapt  during  training,  but  the  pilot 
needs  to  be  weaned  from  the  cue  as  the  skill  level  increases. 

TIME  EXPANSION 

The  entire  simulation  could  be  slowed  down.  This  would  allow  the  pilot  to  explore  the  plant 
dynamics  by  putting  in  a  control  input  and  observing  the  effect  on  the  aircraft’s  attitude  and 
position.  By  slowing  the  entire  simulation  down,  you  provide  the  pilot  the  opportunity  to 
observe  the  sequence  of  effects.  For  example,  as  lateral  stick  is  put  in  the  aircraft  first  banks  and 
then  begins  to  translate.  As  the  bank  increases,  the  pilot  needs  to  add  power  to  maintain  altitude. 

As  another  example,  as  the  pilot  puts  in  rudder  the  aircraft  changes  altitude  unless  the 
appropriate  collective  changes  are  made.  This  approach  is  likely  to  be  most  useful  in  teaching 
the  pilot  the  plant  dynamics.  It  is  not  likely  to  be  appropriate  when  the  pilot  is  trying  to  master 
the  psychomotor  task  itself.  It  would  not  be  appropriate  to  delay  the  effect  of  a  control  input  if 
the  rest  of  the  simulation  were  running  in  real  time.  This  would  increase  the  likelihood  of 
simulator  sickness  and  of  introducing  pilot  induced  oscillation  (PIO). 

WIND  AND  TURBULENCE 

Winds  and  turbulence  can  be  added  to  the  simulation  to  increase  the  difficulty  of  hovering. 
Essentially,  these  winds  disturb  the  aircraft  and  force  the  pilot  to  make  control  inputs  to  maintain 
a  stable  hover.  Early  in  training,  the  pilot  has  more  than  enough  to  do  in  calm  conditions  and 
can’t  cope  with  the  disturbances.  However,  as  skill  increases,  the  pilot  needs  to  recognize 
changes  in  aircraft  position  not  caused  by  control  inputs  and  learn  to  make  the  control  inputs  to 
counteract  the  disturbance.  In  an  ideal  world,  the  pilot  would  be  trained  enough  in  the  simulator 
that  the  winds  in  the  real  world  during  flight,  which  is  almost  never  dead  calm.  (I  assume  that 
they  have  some  weather  standards/minimums  for  initial  flights  and  don’t  go  out  with  students  if 
it  is  blowing  40  with  gusts  to  60!) 

The  application  would  be  to  add  weather  conditions  as  the  pilot  demonstrates  that  he  or  she  is 
able  to  maintain  position  within  criterion. 

/ 

One  of  the  important  things  the  pilot  needs  to  learn  about  winds  is  that  the  aircraft  can  translate 
without  a  change  in  attitude.  A  pure  cross  wind,  for  example,  will  cause  the  aircraft  to  translate 
sideways.  Getting  the  aircraft  to  translate  requires  a  change  in  attitude. 

Constant  winds  require  the  pilot  to  maintain  a  non-standard  attitude.  Simply,  the  aircraft  must  be 
banked  an  appropriate  amount  into  the  wind  to  maintain  position.  As  with  any  change  in  bank 
angle  coordination  with  the  collective  and  rudder  is  necessary. _ _ 
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Gusty  winds  are  similar  to  constant  winds  in  that  they  move  the  aircraft  without  changing  the 
aircraft’s  attitude.  The  difference  is  that  with  the  gusts  the  pilot  must  constantly  make 
corrections,  and  can’t  ever  hope  to  maintain  position  by  setting  the  controls  in  one  position. 

SUMMARY 

This  work  suggests  that  it  is  feasible  to  incorporate  adaptive  training  into  a  hover  trainer  for  use 
in  IERW.  While  we  expect  that  an  adaptive  training  approach  will  ultimately  lead  to  improved 
training  efficiency,  the  literature  provides  little  guidance  on  how  to  adapt  a  psychomotor  skills 
trainer,  under  what  conditions  changes  to  the  trainer  should  be  made,  and  the  expected 
magnitude  of  improvement  in  training  efficiency  expected  from  specific  adaptive  strategies. 
Therefore,  in  our  opinion,  it  is  impossible  to  make  a  reasonable  quantitative  estimation  of  the 
effect  implementing  an  adaptive  training  system  will  have  on  ab  initio  hover  training  at  this  time. 

A  variety  of  approaches  to  developing  models  of  an  Instructor  Pilot  (IP)  and  Instructional 
technologist  (IT)  were  considered,  based  on  this  review  a  Case  Based  Reasoning  approach  is 
recommended.  COTS  tools  develop  and  implement  the  IP  and  IT  models  were  reviewed.  The 
recommended  set  of  tools  consists  of  Easy  Reasoner,  CBRete++,  and  CPR.  Additional  software 
to  support  the  packages  and  integrate  the  system  should  be  written  in  C++. 

The  area  that  present  the  biggest  challenge  to  an  adaptive  hover  training  system  are  determining 
the  prescription  for  each  possible  student  state  and  assessing  the  effectiveness  of  that 
prescription.  The  literature  provides  very  little  guidance  regarding  the  selection  of  adaptive 
features  and  when  they  should  be  employed.  Consequently,  a  major  effort  to  determine  the 
utility  of  various  adaptive  techniques  is  required.  A  set  of  candidate  adaptive  techniques  which 
may  be  prescribed  for  particular  student  states  and  skill  levels  are  described. 
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ACRONYMS 


ARI  Army  Research  Institute 
IEWR  Initial  Entry  Rotary  Wing  Training 
IFT  Intelligent  Flight  Trainer 
IP  Instructor  Pilot 

IT  Instructional  Technology,  Instructional  Technologist 

MTI  Monterey  Technologies,  Inc. 

NVG  Night  Vision  Goggle 

PIO  Pilot  Induced  Oscillation 

SBIR  Small  Business  Innovation  Research  Program 

SP  Student  Pilot 
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PREFACE 


The  following  report  was  prepared,  as  planned,  with  a  focus  on  laying  a  foundation  for 
construction  of  an  Intelligent  Flight  Trainer  (IFT)  for  teaching  rotary-wing  hovering  tasks.  A 
major  re-direction  of  the  project  was  discussed  during  a  project  coordination  held  on  24  May 
2000  at  Monterey  Technologies,  Inc.,  Monterey  CA.  At  that  meeting  the  project  team  was 
informed  that  there  would  be  a  major  change  in  project  direction.  This  report  was  completed  in 
draft  form  prior  to  this  meeting,  and  therefore  the  content  still  reflects  the  study’s  original  intent 
to  develop  an  intelligent  hover  trainer.  The  report  is  incomplete  in  sections  related  to  mission 
definition,  learning  objectives,  and  definition  of  baseline  trainer  design  comparisons.  It  is 
understood  that  the  instructional  design  methods  and  related  training  system  development 
material  presented  herein  still  represent  a  useful  point  of  departure,  with  substantial  application 
to  whatever  new  learning  domain  is  finally  specified  in  the  project  planning  redirection 
instructions. 
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INTRODUCTION 


PURPOSE 


A  review  and  discussion  of  the  complex  skills  training  literature  was  conducted.  The  purpose  of 
this  literature  review  was  to  help  establish  a  foundation  for  the  development  of  An  Intelligent 
Flight  Simulator  for  hover  training.  The  work  was  performed  under  contract  to  Monterey 
Technologies,  Inc.,  Monterey  California  and  was  in  support  of  an  SBIR  contract  (OSD99-04) 
with  the  US  Army  Research  Institute  Rotary  Wing  Aviation  Research  Unit,  Fort  Rucker 
Alabama.  The  scope  of  this,  literature  review  was  limited  to  a  narrow  band  of  skill  acquisition 
and  simulation  literature  that  could  be  accessed  in  the  short  time  period  available  for  the  SBIR 
study.  The  results  and  implications  for  the  design  of  an  adaptive  training  system  deduced  from 
this  body  of  work  are  described  below.  Synopses  of  selected  references  reviewed  during  this 
effort  are  contained  in  Attachment  A.  It  is  recommended  that  a  more  extensive  literature  be 
conducted  if  the  study  progresses  into  a  Phase  II  effort. 

BACKGROUND 

The  US  Army  has  an  interest  in  exploring  emerging  technologies,  including  artificial 
intelligence,  and  their  application  to  advanced  flight  simulation  training.  Methods  to  enhance 
simulation  training  include  the  use  of  student  and  instructor  expert  systems  that  learn  and  adapt 
instruction  to  the  learning  needs  of  a  pilot  undertaking  flight  instruction.  The  basic  concept  is 
that  of  an  “adaptive  trainer”,  which  refers  to  a  training  system  that  uses  student  performance 
during  training  as  a  means  to  adjust  or  adapt  the  level,  or  content  of  instruction  to  improve 
learning  efficiency  or  effectiveness. 

A  previous  study  performed  by  Mulgund,  Asdigha,  and  Zacharias  (1995)  served  as  a  point  of 
departure  in  defining  several  baseline  architectures  for  an  Intelligent  Flight  Trainer  (IFT) 
founded  on  principles  of  adaptive  training.  This  particular  adaptive  trainer  provided  adaptive 
control  of  hover  task  difficulty,  and  a  synthesized  voice  feedback  system  that  provided  flight 
instructional  guidance  in  an  attempt  to  enhance  training. 

Information  gathered  from  this  literature  review,  including  studies  of  human  skill  acquisition, 
flight  simulator  training  effectiveness  experiments,  and  further  discussions  regarding  human 
learning,  the  use  of  performance  feedback,  and  other  instructional  strategies  suggest  that  the 
issues  surrounding  the  use  of  adaptive  training  are  quite  complex.  Some  of  the  findings  in  the 
literature  are  inconsistent  on  key  issues  concerning  the  nature  and  extent  of  feedback  that  should 
be  provided  to  the  learner  during  training,  and  the  permanent  learning  benefit  of  some 
instructional  strategies.  This  report  summarizes  areas  of  agreement  and  disagreement  in  the 
literature  and  discusses  a  number  of  potentially  useful  instructional  methods  that  have  been 
successful  for  teaching  both  cognitive  and  perceptual-motor  components  of  complex  motor 
skills.  Information  provided  here  could  be  a  useful  point  of  departure  for  comparing  various 
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approaches  to  the  use  of  expert  systems  in  simulation  training,  and  for  defining  baseline  training 
system  architecture  that  incorporates  an  adaptive  training  approach. 


ANALYSIS  OF  SELECTED  LITERATURE 
COMPLEX  SKILL  ACOUISTION 


Laboratory  studies  of  complex  skills  have  established  a  useful,  but  limited,  framework  for 
understanding  how  best  to  organize  training  and  teach  flying  skills.  A  good  point  of  departure 
for  discussing  various  teaching  strategies  for  skill  training  is  provided  by  Schneider  (1985).  as 
outlined  below: 

1 .  High-performance  skills  are  characterized  by  three  things: 

•  The  length  of  time  to  become  competent  and  proficient  at  the  task. 

•  A  typically  high  failure  and  non-completion  rate  for  training. 

•  There  is  a  considerable  performance  difference  between  a  novice  and  an  expert. 

2.  Novices  appear  “overtaxed”  and  are  easily  distracted  during  performance,  whereas  the  expert 
appears  to  perform  smoothly  and  without  effort,  and  is  not  easily  distracted. 

3.  As  one  moves  progressively,  from  novice  to  expert,  the  performance  changes  (in  terms  of 
competency  exhibited  and  very  likely  in  terms  of  the  information  processing  and  control 
tasks  performed). 

4.  The  expert  makes  decisions  much  more  rapidly  based  upon  experience,  and  without  much 
“thinking”  or  deliberation  (p.  286). 

During  the  course  of  learning  most  complex  skills,  learners  appear  to  progress  through  distinct 
stages.  There  is  considerable  agreement  in  the  literature  about  the  number  of  stages  (three)  and 
the  events  and  characteristics  that  define  each  stage  (Lane,  1987;  1986). 

Fitts  (1962)  was  one  of  the  first  skill  researchers  to  postulate  these  phases  of  learning  that  he 
called  (1)  Cognitive,  (2)  Fixation,  and  (3)  Autonomous.  One  of  the  most  important  things  that 
Fitts  observed  was  the  early  influence  of  cognitive  processes  on  the  acquisition  of  perceptual- 
motor  skills. 

During  this  cognitive  phase,  learners  attempt  to  “intellectualize”  the  requirements  for  learning 
the  skill,  and  seek  to  establish  some  knowledge  about  the  nature  of  the  task  to  be  performed 
(Fitts,  1962). 

During  the  “fixation  phase”,  the  learner  attempts  to  correct  errors  and  begins  to  focus,  or  fixate 
on  the  most  appropriate  response  patterns  for  accurate  task  performance.  The  final  or 
autonomous  learning  stage  is  characterized  by  a  considerable  improvement  in  timing  and 
coordination,  and  faster  and  more  accurate  performance.  There  is  increased  “automaticity”, 
resistance  to  distractions,  and  a  shift  from  dependence  upon  external  cues  to  internal 
(proprioceptive)  stimuli.  During  later  refinement  of  the  learning  stage  model,  Fitts  and  Posner 
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(1967)  changed  the  term  fixation  to  the  term  associative.  The  term  associative  is  one  that  is 
easily  incorporated  into  contemporary  cognitive  theories  of  learning. 

Stage  learning  models  were  also  proposed  in  the  cognitive  domain.  Anderson  (1982)  formulated 
a  three  stage  learning  model  that  distinguished  between  procedural  knowledge  (represented  by 
production  rules),  and  declarative  knowledge  (represented  by  propositional  networks). 

The  stages  of  learning  proposed  by  Anderson  were  reviewed  and  nicely  summarized  by  Lane 
(1986,  p  132.),  as  follows: 

a.  Declarative.  The  learner  receives  facts,  information,  background  knowledge,  and 
general  instruction  about  a  subject  matter  or  skill.  Mental  elaboration  and  rehearsal  at 
this  stage  helps  to  keep  information  presented  in  working  memory. 

b.  Knowledge  Compilation.  Practice  causes  basic  knowledge  about  a  skill  to  convert 
gradually  from  declarative  form  into  appropriate  new  procedures  that  can  be  applied 
directly  to  the  processing  of  inputs  without  constant  voluntary  attention.  This  stage  is 
comparable  to  Fitt’s  associative  stage  and  represents  formation  of  a  set  of  production 
rules  linking  input  to  output. 

c.  Procedural.  After  declarative  knowledge  is  compiled  into  a  production  system, 
practice  refines  and  strengthens  appropriate  procedures.  Ultimately  responses  reflect 
improved  discrimination  and  generalization  and  become  automatized. 

The  phased  progression  of  learning  into  distinct  stages  by  Fitts  (perceptual-motor  learning)  and 
further  developed  by  Anderson  (cognitive  learning),  strongly  implies  that  the  typical  learner 
transitions  from  a  state  of  knowledge  about  a  subject  or  skill,  to  a  state  of  rule  application.  This 
phased  development  model  has  important  implications  for  the  delivery  and  assessment  of 
instruction  for  both  cognitive  and  perceptual-motor  learning. 

Fitts  and  Posner  (1967),  who  made  the  following  suggestions  recognized  the  need  for  different 
instructional  strategies,  at  initial  and  later  stages  of  perceptual-motor  leaning: 

1 .  During  early  learning  phases, “  call  the  learner’s  attention  to  important  perceptual  cues 
and  response  characteristics,  and  give  diagnostic  knowledge  of  results  (p.  1 1 )”. 

2.  Demonstrations  of  correct  performance  of  the  task  are,  “most  effective  during  early 
learning  phases  (p.l  1)”. 

Most  complex  skill  researchers  agree  that  instructional  strategies  must  adjust  to  the  learners  state 
of  learning,  or  learning  phase.  Schneider  (1984)  stresses  the  importance  of  different  practice 
strategies  for  controlled  processing  and  for  automatic  processing  modes  of  skilled  performance. 
When  a  learner  is  in  early  learning  stages,  and  exhibiting  primarily  controlled  processing,  the 
instructional  strategy  would  be  to  provide  practice  on  wide  variety  of  conditions.  When  learners 
have  reached  the  automatic  processing  skilled  performance  level,  then  they  should  be  presented 
with  consistent  task  elements  repeated  over  many  practice  trials  (Fisk,  Scerbo,  and  Schneider, 
1983). 


SBIR  DATA  RIGHTS  Contract  No.  DASW01-00-M-4084.  Contractor  Name:  Monterey  Technologies,  Inc.,  Address:  26400  Silver  Cloud  Ct.,  #103, 
Monterey,  CA  93940.  Expiration  of  SBIR  Data  Rights  Period:  July  1 . 2005.  The  Government's  rights  to  use,  modify,  reproduce,  release,  perform, 
display,  or  disclose  technical  data  or  computer  software  marked  with  this  legend  are  restricted  during  the  period  shown  as  provided  in  paragraph 
(b)(4)  of  the  Rights  in  Noncommercial  Technical  Data  and  Computer  Software- Small  Business  Innovation  Research  (SBIR)  Program  clause 
contained  in  the  above  identified  contract.  No  restrictions  apply  after  the  expiration  date  shown  above.  Any  reproduction  of  technical  data, 
computer  software,  or  portions  thereof  marked  with  this  legend  must  also  reproduce  the  markings 


Appendix  1  -  6 


Monterey  Technologies,  Inc 


Contract  No.  DASW0I-00-M-4084 


SBIR  Phase  I  Final  Report 


Instructional  psychologists  have  incorporated  some  of  the  findings  from  the  complex  skill  studies 
in  their  theories  of  instruction  design.  Gagne’  and  Briggs  (1979),  for  example  derive  their 
instructional  presentation  methods  for  the  cognitive  domain  based  on  some  of  the 
aforementioned  human  learning  and  information  processing  principles. 

In  answer  to  the  question,  “How  is  the  student  coaxed  along  during  instruction”,  Gagne’  and 
Briggs  outline  some  of  the  critical  events  of  instruction  as  follows: 

1 .  Gaining  attention 

2.  Informing  the  learner  of  the  objective 

3.  Stimulating  recall  of  prerequisite  instruction 

4.  Presenting  stimulus  material 

5.  Providing  learning  guidance 

6.  Eliciting  the  performance 

7.  Providing  feedback  about  performance  correctness 

8.  Assessing  the  performance,  and 

9.  Enhancing  learning  and  transfer  (p.  157) 

Merrill  (1980,  1983)  believed  that  the  approach  taken  by  Gagne’  and  Briggs  was  in  the  right 
direction,  but  suffered  from  a  lack  of  specificity.  He  proposed  a  Component  Display  Theory 
(CDT)  of  instruction  that  represented  a  systematic  and  detailed  procedural  framework  for 
teaching  cognitive  skills.  Merrill’s  instructional  strategies  included  procedures  for  defining  and 
classifying  cognitive  tasks,  and  then  systematically  organizing  instruction  around  the 
classification  of  task-based  learning  objectives.  The  instructional  design  theories  proposed  by 
Gagne’,  Briggs,  and  Merrill  are  sometimes  referred  to  as  “prescriptive”  instructional  theories 
because  they  take  defined  learning  outcomes  (typically  specified  as  learning  outcome  objectives) 
and  recommend  a  particular  instructional  presentation  and  assessment  method. 

Prescriptive  Instructional  Methods  for  the  Cognitive  Domain 

A  prescriptive  approach  to  instruction  represents  an  attempt  to  determine  the  optimum  method 
for  teaching  and  assessing  specific  learning  outcomes.  Glaser  and  Resnick  (1972)  outline  some 
of  the  key  characteristics  of  a  prescriptive  approach  as  follows: 

1 .  Description  of  the  state  of  learning  to  be  achieved. 

2.  Description  of  the  initial  state  of  the  learner 

3.  Actions  that  can  be  taken  to  transform  the  initial  state  into  the  desired  state. 

4.  Assessment  of  the  transformation  of  the  state  to  determine  learning  results  of  each 
(instructional)  action  taken. 

5.  Evaluation  of  the  attainment  of  the  terminal  state  desired  (p.  208). 

Prescriptive  methods  of  instruction  grew  out  of  dissatisfaction  with  attempts  to  directly  apply 
general  principles  of  learning  that  were  developed  in  the  psychological  laboratory.  Gagne’ 
(1962),  for  example,  found  few  of  the  commonly  accepted  laboratory  learning  principles,  such  as 
response  reinforcement  and  distribution  of  practice,  to  be  useful  in  practical  learning  situations. 
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Gagne’  formulated  one  of  the  first  prescriptive  theories  of  instruction,  base  upon  his  study  of 
military  training  in  late  50’s  and  60’s. 

His  recommendation  about  organizing  instruction,  outlined  below,  became  the  hallmark  of 
prescriptive  theory: 

1 .  Analyze  component  tasks. 

2.  Specify  method  for  teaching  each  component  task  (of  a  particular  skill). 

3 .  Specify  instructional  strategy,  presentation  sequence  and  select  media  most 
appropriate  for  each  phase  of  training.  (Adapted  from  Gagne’,  1962,  p.  86). 

Categories  of  Learning 

An  organizing  principle  of  prescriptive  theories  of  instruction  is  the  classification  of  different 
learning  outcomes.  Robert  Gagne’  (1965;  1985)  was  one  of  the  first  instructional  psychologists 
to  propose  that  different  learning  outcomes  require  different  learning  methods  or  strategies. 

Table  1  shows  his  later  description  of  various  learning  outcomes  proposed  by  Gagne’  (Wagner 
and  Gagne’,  1988).  The  examples  given  for  each  learning  outcome  listed  were  edited  for 
purposes  of  clarity  to  this  analysis  and  report. 

Table  2  shows  some  of  the  recommended  learning  prescriptions,  or  teaching  guidelines,  for 
selected  learning  outcomes,  based  on  the  theory  of  instruction  developed  by  Gagne’,  as  discussed 
in  Wagner  and  Gagne’  (1988).  The  guidelines  provided  in  the  Gagne’  model  has  received  some 
criticism  for  being  far  too  general  to  be  useful  in  most  practical  learning  situations.  Merrill 
(1983;  1988)  further  developed  ideas  originated  by  Gagne’  to  formulate  a  more  detailed 
procedural  framework  for  instruction  design,  referred  to  as  Component  Display  Theory  or  CDT. 

TABLE  1:  Categories  of  Human  Learning 

(Wagner  and  Gagne’,  1988  p.  37) 

Example  of  Performance 

Use  symbols  and  language  to: 

Sort  squares  and  triangles 

Classify  buildings  given  definitions 

Apply  process  to  reduce  fractions 
Create  math  formula  derivation 
Create  a  way  to  learn  a  new  skill 
State  historical  facts 
Steer  an  automobile  or  an  aircraft 
Choose  a  favorite  person  or  movie 


Learning  Outcome 
Intellectual  Skill 
Concrete  Concept 
Defined  Concept 
Rule 

Higher-order  Rule 
Cognitive  Strategy 
Verbal  Information 
Motor  Skill 
Attitude 
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Table  2:  Examples  of  Prescriptive  Instructional  Guidelines  for  Cognitive  Domain 


Strategy 

(Adapted  from  Reigeluth,  1983,  p.  94) 

Intellectual  Skill  Factual  Information 

Motor  Skill 

1 .  Gain  Attention 

(Draw  attention  by  highlighting  or  changing  sensory  mode) 

2.  Tell  Objectives 

Show  learner 

Show  subject 

Present  key  skill 

performance 

outline  or  organizer 

components 

3.  Stimulate  Recall 

expected 

Provide  review  or 

Show  Subject 

Demonstrate 

summary  of  core 

outline  or  organizer 

performance 

4.  Present  Instruction 

ideas 

Show  examples 

State  facts. 

Provide  simulated 

and  how  to  do 

concepts,  etc. 

practice 

5.  Provide  Guidance 

Verbal  prompt 

Link  subject  to 

Provide  feedback  or 

and  aids 

related  material 

guidance 

6.  Elicit  Performance 

Ask  for  solution 

Ask  for  recall  of 

Ask  learner  to 

7.  Provide  Feedback 

Show  correct 

information 

Confirm  right 

perform 

Provide  corrective 

solution 

answer(s) 

feedback 

8.  Assess  Performance 

Learner  shows 

Learner  takes 

Learner  performs 

how  to  do 

written  test 

task 

9.  Enhance  Transfer 

Give  variety  of 

Provide  links  to 

Give  more  practice 

practice  problems 

related  materials 

on  task 

Merrill  and  his  associates  (Merrill  1983;  Merrill,  Reigeluth,  and  Faust,  1979)  developed  a 
prescriptive  instructional  theory  using  Gagne’  as  a  point  of  departure.  The  so-called  component- 
display  theory  proposed  by  M.  David  Merrill  focused  primarily  on  methods  designed  to  ensure 
high  quality  instruction  for  cognitive  skills.  Merrill  believed  that  the  work  by  Gagne’  and  other 
early  instructional  theorists  did  much  to  help  systematize  instructional  methods  but  left  too  much 
open  to  interpretation  by  instructional  designers. 

Component  Display  Theory  (CDT)  methods  address  specific  micro  strategies  for  optimizing 
instruction  in  the  cognitive  domain.  The  principal  behind  CDT  is  to  arrange  instruction  in 
accordance  with  specific  prescriptions  for  each  of  several  categories  of  learning  outcomes. 
Cognitive  learning  outcomes,  for  example,  are  classified  in  two  dimensions:  (1)  the  type  of 
subject  matter  (facts,  concepts,  principles,  and  procedures);  and  (2)  the  desired  performance  level 
on  the  learning  task  (remember,  use,  find).  A  more  detailed  background  description  of  CDT,  as 
well  as  instruction  design  guidelines  and  examples  is  presented  in  Ciavarelli  (1988),  and  Merrill, 
Reigeluth,  and  Faust  (1979).  Facts  are  arbitrary  ideas  or  events;  procedures  are  sequential  steps 
required  to  perform  a  specific  operation;  principles  are  cause  and  effect  relationships;  and 
concepts  represent  classification  of  things  according  to  common  attributes.  The  remember,  use, 
and  find  categories  respectively  memorization,  skill  application,  and  cognitive  strategy 
performance  levels. 
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Table  3  shows  the  two-dimensional  (task  X  performance  level)  classification  matrix  commonly 
used  in  CDT. 


TABLE  3 

Classification  of  Learning  Outcomes  for  the  Cognitive  Domain 


FACTS 

CONCEPTS 

PRINCIPLES 

PROCEDURES 

FIND 

(Discover) 

The  teaching  method  used  in  CDT  typically  incorporates  both  primary  and  secondary 
instructional  presentation  strategies  as  discussed  below. 


Primary  Presentations 

Merrill’s  CDT  suggests  that  instruction  be  organized  around  a  series  of  presentations,  or 
displays.  Primary  presentations  are  represented  by  information  presented  to  the  learner  in  terms 
of  “telling”  or  “questioning”  (Tell  or  Question),  as  depicted  in  Table  4.  The  instructor  has  a 
choice  of  presenting  “generalities”  of  the  subject  matter  or  presenting  specific  examples  or 
instances. 


TABLE  4. 

organization  of  Instruction  Around  Presentation 


TELL 

QUESTION 

GENERALITY 

EXAMPLE 

If  we  were  attempting  to  teach  a  student  the  differences  between  a  fighter  aircraft  and  an  attack 
bomber,  for  example,  we  would  define  the  key  attributes  that  distinguish  the  two,  then  show 
some  examples  of  each.  So  we  can  define  attributes  (Tell/Generality),  or  we  can  present 
examples  (This  is  example  of  a  fighter  jet;  This  is  example  of  an  attack  jet).  The  same  material 
can  be  presented  using  a  question  method.  We  can  ask  the  student  to  define  the  attributes,  or 
select  correct  attribute  definition,  or  we  can  show  an  example  (picture  of  either  aircraft)  and  ask 
the  student  to  correctly  identify  the  example  shown. 


Secondary  Presentations 
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Secondary  presentations  are  strategies  inserted  into  the  learning  situation  that  help  the  learner  to 
assimilate  the  subject  matter.  Some  of  the  more  common  “secondary”  instructional  presentations 
listed  below. 

•  Advance  Organizers  (organization  charts,  subject  matter  maps,  models,  graphs,  pictures  that 
illustrate  key  points  and  structure  of  learning  material) 

•  Isolation  and  Highlighting  (coloring,  pointing  to,  or  otherwise  identifying  key  points  or 
salient  ideas  or  cues) 

•  Semantic  Labels  (pointing  out  meaningful  connections  and  relationships  in  the  knowledge 
structure) 

•  Verbal  Mnemonics  (rhymes,  limericks,  slogans,  and  other  memory  enhancing  methods) 
Assessment  ('Written  or  Oral  Exams) 

Various  forms  of  testing  are  used  to  assess  the  competency  or  achievement  level  of  the  learner, 
including  written  tests  and  performance  demonstrations.  The  test  or  performance  demonstration 
is  developed  to  assess  a  particular  performance  level  specified  in  the  learning  objective,  such  as: 

•  Memory  Level  (remember) 

•  Application  Level  (use) 

•  Problem  Solving  (find) 

In  general,  CDT  instructional  presentations  follow  the  format  summarized  below: 

1 .  A  “rule”  statement  related  to  the  instructional  content  is  presented,  such  as  a  brief 
factual  statement,  definition  of  a  concept  or  concept  class,  description  of  a  principle, 
or  listing  of  steps  in  a  procedure. 

2.  Following  the  rule  statement  (s)  related  to  the  core  content,  some  examples  of 
application  are  presented.  For  example,  in  a  procedure  objective,  the  correct  steps  in 
the  procedure  would  be  stated  together  with  the  information  about  an  operational 
conditions,  cues,  and  expected  system  responses  to  control. 

3.  After  presentation  of  one  or  more  examples,  instances  or  demonstrations,  the  learner 
would  be  given  a  opportunity  to  recall  the  correct  procedure,  concept  definition,  or 
key  principles  taught  in  the  lesson. 

4.  Following  the  recall  session,  the  learner  would  be  given  a  opportunity  to  practice 
(apply  or  use)  the  procedure,  practice  classifying  concept  classes,  or  solve  problems 
using  specified  principles,  before  being  tested  on  content  and  application.  The 
practice  exercises  should  provide  a  sufficient  range  of  experience  to  meet  expected 
competencies  defined  in  the  learning  objectives.  During  practice  sessions  the  learner 
is  often  given  diagnostic,  corrective  feedback  and  other  guidance  to  facilitate 
learning.  The  guidance  is  eventually  reduced  or  withdrawn  as  the  learner  progresses 
and  prepares  for  competency  testing. 

5.  The  student  is  given  a  competency  test  (without  any  instructional  aids)  to  measure 
learning  progress,  and  to  determine  the  need  for  additional  instruction.  The  test  may 
be  in  written  or  oral  form,  and  would  be  conducted  in  the  reference  to  the 

_ performance  level  specified  in  the  learning  objectives. _ 
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By  way  of  summary,  a  typical  application  of  CDT  calls  for  analysis  of  learning  tasks,  preparation 
of  specific  learning  objectives  and  classifying  them  as  to  type  task  and  performance  level,  and 
then  arranging  learning  in  a  way  that  optimizes  instruction  for  a  particular  task  classification. 

For  example,  in  developing  a  concept  lesson  using  CDT,  the  instruction  designer  would  arrange 
instruction  as  follows: 

Objective:  Sort  all  aircraft  examples  provided  into  fighter  jet,  patrol,  transport,  attack  helicopter, 
troop  transport  helicopter. 

Performance:  Use  concept  (sort  by  defined  attributes) 

Instructional  Sequence: 

1 .  Present  Tutorial  that  defines  the  concept  classes  and  their  attributes. 

2.  Present  examples  with  concept  labels  (pictures  with  identifiers) 

3.  Item  1  and  2  may  be  completed  simultaneously. 

4.  Provide  learner  opportunity  to  recall  concept  class  attributes  and  practice  sorting  into 
correct  classes  with  immediate  corrective  feedback  of  results. 

5.  Test  student  at  completion  of  practice  trials  at  appropriate  level  of  performance 
defined  in  the  learning  objective. 

In  addition  to  arranging  the  “primary”  instructional  presentations  as  described  above,  CDT  often 
uses  “secondary”  presentations  to  facilitate  learning.  Secondary  presentation  methods  include 
prompting  and  corrective  feedback,  highlighting  key  points  and  critical  information,  and 
providing  knowledge  organizational  frameworks,  such  as  flow  charts,  block  diagrams, 
illustrative  animations  and  simulations.  Merrill  (1983)  believed  that  such  “secondary” 
instructional  presentations,  or  “displays”,  help  to  elaborate  the  primary  instruction  and  thereby 
facilitate  the  learning  process. 

CDT  attempts  to  establish  instructional  quality  providing  some  guidelines  that  can  be  used 
during  training  development  or  for  assessing  the  quality  of  established  instruction. 

The  instructional  quality  guidelines  include  ground  rules  for  checking  the  adequacy  of 
instructional  objectives  (correct  format  and  content),  for  checking  the  consistency  between  the 
learning  objective  and  instructional  delivery  strategy,  and  between  learning  objective  and 
associated  assessment  method  (Merrill,  1983). 

Finally,  CDT  accepts  that  individual  learning  differences  are  inevitable  and  handles  differences 
in  ability  and  learning  style  by  allowing  the  learner  to  control  the  learning  process  (selection  of 
instructional  presentations,  the  pace  of  instruction,  and  the  extent  of  practice  sessions  to  be 
completed  before  competency  testing). 

CDT,  as  well  as  the  prescriptive  methods  of  instruction  developed  by  Gagne’,  Merrill,  Briggs 
and  others,  attempted  to  address  the  issues  of  developing  all  forms  of  instruction.  These  methods 
are  founded  on  an  instruction  design  process  founded  upon  cognitive  learning  principles.  Basic 
precepts  of  the  prescriptive  approach  are: 
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1 .  Recognition  that  various  learning  objectives  represent  distinct  learning  outcomes  that 
may  require  different  teaching  strategies  and  presentation  methods. 

2.  Improved  understanding  of  cognitive  processing  that  underlies  knowledge  acquisition 
and  skilled  performance  in  any  specific  learning  domain. 

3.  Use  of  explicit  methods  to  promote  learning  by  providing  features  of  instruction, 
including  performance  knowledge  organizers  and/or  task  information  templates, 
verbal  prompting  and  immediate  performance  feedback  and  knowledge  of  results,  to 
help  the  learner  more  efficiently  process  and  acquire  the  knowledge  and  skills  desired 

Prescriptive  methods  of  instruction  have  focused  almost  exclusively  on  the  cognitive  domain. 
Instructional  prescriptions,  for  complex  skills  have  not  been  as  well  developed,  so  one  must  look 
to  a  body  of  scientific  knowledge  accumulated  largely  from  laboratory  studies  in  human 
performance.  It  is  presumed  that  prescriptive  methods  can  be  derived  from  such  studies,  if 
reliable  and  valid  conclusions  can  be  drawn  from  the  research  in  human  performance  and 
learning. 

Methods  for  Improving  Instruction  of  Perceptual-Motor  Skills 

Much  of  the  early  literature  in  perceptual-motor  learning  was  heavily  influenced  by  behavioral 
methodology  developed  in  the  psychology  laboratory,  with  concentration  in  such  areas  as 
practice  length  and  distribution,  sensory  feedback  and  knowledge  of  results,  and  various  methods 
of  response  reinforcement  (Bilodeau,  1966,  Adams,  1968).  There  was  a  definite  shift  in  focus 
from  the  behavioral  to  the  cognitive  approach  to  the  study  and  understanding  of  complex 
perceptual-motors  skills  beginning  in  the  late  sixties. 

Fitts  and  Posner  (1967),  for  example,  postulated  that  there  were  “sensor-motor”  schemas  that 
served  as  organizers  and  controllers  of  complex  skills.  These  schemas  were  thought  to  operate 
much  like  a  stored  computer  program  and  their  operation  accounted  for  the  so-called 
automatization  of  complex  skills.  Fitts  and  Posner  also  recognized  the  important  influence  that 
cognitive  training  for  learning  the  structure  of  the  perceptual-motor  skill,  and  the  importance  of 
corrective  feedback  of  performance  during  practice  trials. 

Gibson  (1966),  who  had  a  very  prominent  influence  in  the  area  of  perceptual  learning  for 
aviators,  observed  that  pilots  during  early  flight  training  were  easily  overwhelmed  by  a  cascade 
of  “spinning  instruments”,  but  eventually  learned  to  process  and  interpret  both  instrument  and 
out  of  cockpit  cues  with  ease.  He  postulated  a  theory  of  perceptual  learning  based  upon  the 
experience  gained  while  practicing  the  task  and  observing  the  relationships  between 
environmental  cues  and  successful  responses  to  cues  perceived  as  relevant  to  aircraft  control  or 
flight  status.  The  learner,  in  effect,  while  practicing  a  complex  perceptual  motor  task,  gradually 
learns  what  cues  are  important  to  attend  to.  Gibson  suggested  those  experienced  pilots,  or 
instructors,  could  speed  this  learning  process  by  highlighting  the  important  cues,  and 
demonstrating  the  relation  between  cues  and  responses  by  the  learner. 
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Merrill  (1971)  places  emphasis  on  the  importance  of  perceptual  discrimination  and  correct 
response  chaining  during  complex  perceptual-motor  learning.  The  chaining  of  response 
sequences  are  combined  to  form  the  required  tie  between  perception  and  movement  behaviors 
required  for  skilled  performance.  In  order  to  promote  learning,  he  recommends  prompting  the 
correct  response,  providing  knowledge  of  results,  reinforcing  correct  responses  and  response 
topography,  making  cues  more  distinctive,  use  of  verbal  prompting,  and  demonstration  of  a  skill 
performed  by  an  expert. 

Bailey,  Hughes  and  Jones  (1980)  were  successful  in  using  the  behavioral  method  of  “backward 
chaining”  part-task  training  while  teaching  US  Air  Force  pilots  dive  bombing.  This  technique 
was  also  used  successfully  by  Wightman  (1983)  to  teach  Navy  pilots’  carrier  landing  skills.  The 
backward  chaining  approach  involves  teaching  the  terminal  segment  of  a  sequential  task  first, 
and  then  backing  through  the  task  sequence  until  all  task  components  have  been  mastered.  This 
approach  is  accepted  practice  by  many  trainers,  and  fits  well  with  tasks  like  carrier  landing  and 
dive  bombing  which  are  clearly  sequential  and  easily  segmented  for  a  part-task  training 
approach. 

Various  methods  of  performance  feedback  have  been  studied  as  a  means  to  improve  performance 
and  learning  of  perceptual  motor  skills.  These  studies  include  those  that  provide  concurrent 
feedback  to  learners  during  skill  acquisition,  and  those  that  provide  after  action  knowledge  of 
results.  The  influence  of  feedback  on  learning  complex  perceptual-motor  skills  as  been  very 
inconsistent  and  is  sometimes  difficult  to  interpret  or  apply  directly  to  training.  Schmidt  and 
Wulf  (1997)  reviewed  literature  on  the  topic  of  concurrent  feedback  and  conducted  experiments 
designed  to  improve  our  understanding  of  perceptual-  motor  learning  under  various  feedback 
conditions.  Subjects  performed  a  laboratory  task  (lever  positioning)  with  visual  feedback,  when 
provided.  The  researchers  were  interested  in  a  better  understanding  of  many  studies  that  showed 
that  concurrent  feedback  often  helped  performance  during  task  practice  trials,  but  the 
performance  gains  were  temporary  and  did  not  transfer  to  the  operational  task  when  feedback 
was  removed.  In  other  words,  the  feedback  affected  performance  but  did  not  have  a  permanent 
effect  on  learning.  One  possible  explanation  for  this  finding  is  that  the  concurrent  feedback 
essentially  functions  as  a  “crutch”  used  to  produce  the  correct  responses  during  practice  trials, 
and  when  removed  the  learner  can  no  longer  perform  the  task  at  the  required  level.  Another 
explanation  for  the  lack  of  “learning”  transfer  is  that  the  learners’  attention  is  focused  on  the 
augmented  feedback  and  therefore  may  fail  to  concentrate  on  the  most  important  relationships 
between  naturally  occurring  cues  and  the  associated  correct  responses.  Experiments  performed 
by  Schmidt  and  Wulf  (1997)  supported  findings  of  earlier  studies  by  Schmidt  (1991),  and 
Schmidt,  Bjork  (1992)  which  showed  performance  gains  for  concurrent  feedback  during  practice 
trials,  but  no  learning  effects. 

The  current  study  by  Schmidt  and  Wulf  concluded  that  use,  or  misuse,  of  concurrent  feedback 
during  some  forms  of  perceptual -motor  learning  may  actually  impair  learning.  Their  findings 
seem  to  match  others  who  have  studied  the  complex  effects  of  feedback  and  knowledge  of 
results  on  perceptual-motor  learning. 
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Salomoni,  Schmidt,  and  Walter  (1984),  after  conducting  a  comprehensive  literature  of  laboratory 
studies  of  results  and  motor  learning,  concluded  that  the  effects  of  performance  feedback  in 
many  cases  drop  out  following  the  acquisition  phase  of  learning.  In  other  words,  feedback 
provided  during  practice  may  have  temporarily  enhanced  performance,  but  the  advantage  was 
not  sustained  in  follow  on  learning  transfer  and  retention  tests.  Similarly,  Nishikawa  (1985) 
found  that  many  of  the  immediate  feedback  studies  he  reviewed  indicated  that  immediate 
feedback  during  practice  trials  mostly  affected  performance  but  had  little  or  no  impact  on  the 
eventual  learning  of  a  complex  skill.  He  did  find  that  some  forms  of  feedback  were  effective, 
but  the  effectiveness  depended  upon  the  type  of  task,  the  nature  of  the  feedback,  and  the  learning 
state  of  the  learner  at  the  time  feedback  was  provided.  Cohen  (1985)  shed  some  light  on  the 
inconsistency  of  feedback. application  by  suggesting  that  feedback  is  most  useful  during  early 
learning  phases,  and  that  one  should  consider  the  timing  and  content  of  feedback  required. 

Cohen  believed  that  feedback  in  computer-aided  instruction  should  let  the  student  know  whether 
a  given  response  is  right  or  wrong,  and  also  provide  information  about  how  to  locate  and  correct 
errors. 

Various  methods  for  practicing  key  components  of  a  complex  skill  have  been  attempted  to 
improve  learning  and  retention.  Wightman  and  Lintem  (1985)  reviewed  much  of  the  literature 
on  part-task  training  and  established  a  clear  framework  relating  the  types  of  part-task  variations 
and  methodologies  for  separating  performance  from  learning  effects.  These  authors  concentrated 
on  manual  control  task  studies  and  various  methods  for  decomposing  tasks,  as  defined  below: 

•  Segmentation,  which  is  partitioning  components  of  a  sequential  skill 

•  Fractionation,  which  is  separating  elements  of  a  time-shared  task 

•  Simplification,  which  is  making  difficult  tasks  easier  by  slowing  pace  or  reducing 
information  flow 

•  Augmented  feedback,  which  is  highlighting  or  adding  supplementary  cues  to  stimulus 
environment 

Wightman  and  Lintem  concluded  that  the  only  consistent  positive  results  were  obtained  using 
the  segmentation  method,  such  as  the  backward  chaining  techniques  used  by  Bailey,  et  al.  (1980) 
cited  earlier.  Wightman  (1983)  used  the  backward-chaining  method  and  showed  positive 
transfer  of  learning  to  an  aircraft-landing  task.  Fisk,  Scerbo,  and  Schneider  (1983)  and 
Schneider  (1985)  argue  that  training  on  a  whole  task  in  a  “natural”  setting  is  inefficient  in  many 
cases  because  the  real  world  does  not  typically  expose  the  learner  to  consistent  task  elements. 
These  authors  recommend  (1)  isolating  consistent  task  elements  for  focused  practice  (2) 
providing  an  opportunity  for  repeated  practice  with  feedback,  (3)  practicing  critical  task 
components  separately  to  reduce  information  overload,  (4)  training  to  an  acceptable  level  of 
accuracy,  and  (5)  incorporating  extrinsic  motivating  conditions  and  learning  incentives  (isolating 
consistent  task  elements  for  focused  practice). 

Another  strategy  for  teaching  complex  skills  is  to  adapt  the  difficulty  of  a  task  to  the  learner’s 
skill  level.  During  learning  acquisition  trials,  the  task  is  simplified  through  such  techniques  as 
slowing  the  required  response  time,  or  made  easier  by  providing  augmented  feedback,  verbal 
prompting  or  cue  enhancement.  This  method  is  referred  to  as  adaptive  training.  Adaptive 
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training  originated  in  studies  of  manual  tracking  (Kelly,  1969),  and  has  been  proposed  as  a 
means  to  accommodate  individual  differences  in  learning  rate,  as  well  as  for  providing 
differential  feedback  at  various  learning  stages  during  skill  acquisition. 

The  adaptive  training  process  is  one  in  which  the  presentation  of  instruction  is  varied  in 
accordance  with  student  learning  rate.  A  computer  algorithm  determines  presentation  rate  that  is 
governed  so  that  task  speed,  or  difficulty,  progressively  increases  with  improved  task 
performance.  In  a  study  of  perceptual-motor  skill  acquisition,  Mane’  (1984),  compared  the 
effectiveness  of  adaptive  training  to  a  part-task  training  method.  The  task  studied  was  a  video 
“space  fortress”  game  in  that  simulated  a  surface  ship  warfare  mission,  including  tactical 
situation  and  weapon  launch.  Mane’  used  a  slowed  down  version  of  the  task  for  beginners  as  an 
adaptive  training  strategy,  increasing  the  speed  as  learners  progressed. 

A  second  part-task  strategy  was  also  tested  in  his  experimental  framework.  He  allowed  this 
treatment  group  to  practice  critical  elements  of  the  task  separately  prior  to  transfer  to  the  whole 
task.  A  control  group  practiced  the  whole  task  at  a  constant  (normal)  speed.  Mane’  held  the 
view  that  part-task  training,  exemplified  breaking  the  task  into  basic  components  and  then 
allowing  the  learner  to  practice  each  task  separately,  enabled  learners  to  concentrate  on  important 
variables  associated  with  successful  performance. 

The  results  of  Mane’s  experiments  clearly  and  significantly  showed  the  superiority  of  the  part- 
task  method  in  his  particular  study  application.  In  another  similar  study,  Wightman  and  Sistrunk 
(1987)  studied  the  use  of  adaptive  and  part-task  training  strategies  on  an  aircraft  carrier  landing 
task,  using  task  simplification  or  progressive  difficulty  and  a  part-task  strategy  (segmentation). 
Wightman  and  Sistrunk  reported  that  there  was  significant  advantage  for  both  low  and  high 
aptitude  subjects  using  the  part-task  segmentation  method,  compared  to  the  adaptive  training 
(progressive  difficulty)  method. 

It  may  be  postulated  that  motor  skill  learning,  like  cognitive  learning,  is  founded  on  the 
development  of  “generalized  schema”  that  govern  the  transfer  process  and  affect  the  speed  of 
learning  new  skills.  This  postulate  implies  that  systematic  variation  in  task  demands  and  the 
stimulus  environment  during  training  trials  help  to  establish  and  solidify  developing  schema  for 
improved  perceptual-motor  acquisition  and  retention.  A  number  of  studies  following  Mane’s 
1984  study,  using  the  Space  Fortress  video  game  appear  to  support  this  reasoning.  Gopher, 

Weil,  and  Bareket  (1994)  summarize  the  findings  of  these  studies,  and  present  some 
experimental  evidence  that  general  attention  control  strategies  can  be  trained  using  video  games, 
like  Space  Fortress,  and  that  such  pre-training  shows  positive  transfer  to  flight  training  tasks 
performed  in  a  flight  simulator  and  aircraft.  In  this  case,  trainees  practice  handling  information 
overload  and  to  establish  successful  task  prioritization  schemes  that  work  well  in  performing  the 
tactical  tasks  of  the  Space  Fortress  game  (moving  a  vehicle  to  gain  tactical  advantage,  and 
accurately  firing  simulated  weapons).  Results  of  the  study  showed  that  students  trained  on  the 
video  game  had  a  clear  advantage  during  later  flight  training. 


SBIR  DATA  RIGHTS  Contract  No.  DASW01-00-M-4084.  Contractor  Name:  Monterey  Technologies,  Inc.,  Address:  26400  Silver  Cloud  Ct.,  #103, 
Monterey,  CA  93940.  Expiration  of  SBIR  Data  Rights  Period:  July  1 . 2005.  The  Government's  rights  to  use,  modify,  reproduce,  release,  perform, 
display,  or  disclose  technical  data  or  computer  software  marked  with  this  legend  are  restricted  during  the  period  shown  as  provided  in  paragraph 
(b)(4)  of  the  Rights  in  Noncommercial  Technical  Data  and  Computer  Software- Small  Business  Innovation  Research  (SBIR)  Program  clause 
contained  in  the  above  identified  contract.  No  restrictions  apply  after  the  expiration  date  shown  above.  Any  reproduction  of  technical  data, 
computer  software,  or  portions  thereof  marked  with  this  legend  must  also  reproduce  the  markings. 


Appendix  1-16 


Monterey  Technologies,  Inc. 


Contract  No  DASW01-00-M-4084 


SBIR  Phase  I  Final  Report 


Recently  researchers  have  begun  identifying  the  neurological  foundations  for  motor  skills  and 
motor  skill  learning.  Willingham  (1999),  for  example,  reported  findings  from  neurological 
experiments  and  case  studies  (some  done  with  positron  emission  imaging  of  the  brain  anatomy) 
that  show  underlying  brain  structures  involved  in  different  levels  of  motor  response  processing. 
Willingham  supports  views  regarding  underlying  information  processes  that  include  (1)  a 
strategic  (thinking  and  goal  selection)  neural  process,  (2)  a  perceptual-motor  integration  process, 
(3)  a  motor  response  sequencing  process,  and  (4)  a  dynamic  process  that  that  accounts  for 
learning  new  motor  actions  associated  with  perceptual  cues.  He  then  identifies  hierarchical 
neurological  structures  that  form  the  substrate  for  both  conscious  and  unconscious,  or  automatic, 
perceptual-motor  control  actions. 

Adaptive  training  is  considered  another  form  of  feedback  to  the  learner  during  application,  and 
therefore  poses  some  of  the  same  issues  regarding  the  content,  timing,  and  format  of  such 
feedback  options  and  their  effects  on  the  learning  process.  Application  of  various  forms  of 
feedback,  including  knowledge  of  results,  augmented  cueing,  verbal  prompting,  task  difficulty 
adjustments,  etc.,  appear  to  play  a  multifunctional  role,  acting  to  motivate,  reinforce,  and  to 
inform  the  learner.  Effective  use  of  feedback,  however,  depends  on  the  nature  of  the  task,  the 
state  of  the  learner,  and  the  method  and  timing  of  delivery.  It  appears  from  the  evidence,  that 
augmented  feedback  during  practice  of  complex  skills  can  improve  learning.  But  only  if  it  is 
applied  on  tasks  known  to  depend  on  external  cueing  for  response  guidance,  if  learners  have  not 
attained  mastery,  and  if  used  sporadically,  and  if  augmented  feedback  is  eventually  is  withdrawn 
to  reduce  dependency  prior  to  transfer  to  the  operational  task. 

It  is  clear  from  the  literature  that  conditions  that  might  improve  performance  during  practice  do 
not  necessarily  result  in  better  learning.  Variables  such  as  distributed  practice,  sessions,  extra 
practice  opportunities  (over  learning),  and  augmented  feedback  have  had  mixed  results  regarding 
their  permanent  effects  on  learning  as  measured  by  learning  retention  and  transfer  (Swezey  and 
Llaneras,  1997). 

Various  approaches  have  been  taken  to  improve  perceptual -motor  training  using  cognitive 
instruction.  Singer,  Korienek,  and  Ridsdale  (1980)  investigated  the  effectiveness  of  various 
cognitive  strategies.  These  strategies  included  imagery  rehearsal,  chunking,  rote  verbalization, 
and  informed  choice  (p.97).  The  serial  manipulation  task  studied  required  the  learner  to 
correctly  perform  a  sequenced  response  selection,  like  lever  positioning  that  was  followed  by 
feedback.  In  the  chunking  condition,  subjects  were  instructed  to  group  responses  into  a  sequence 
of  three.  For  the  rote  memorization  treatment,  subjects  performed  the  task  sequence  while 
verbalizing  their  responses. 

In  the  imagery  condition,  subjects  were  asked  to  mentally  picture  responses  that  are  logically 
categorized  into  storage  bins.  Informed  choice  consisted  of  one  or  all-previous  strategies,  self- 
selected  by  subjects.  A  control  group  received  no  instructions.  None  of  the  purported  cognitive 
strategies  appeared  to  support  the  hypothesis  that  predicted  better  performance  of  chunking  and 
imagery  methods  over  rote  verbalization.  The  authors  discussed  the  difficulties  of  defining  and 
testing  such  strategies,  based  upon  theoretical  instructional  expectancies. 
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On  the  other  hand,  some  cognitive  instructional  strategies  have  resulted  in  improved  perceptual- 
motor  training.  Landeweed,  Seegers,  and  Perryman  (1981)  found  the  use  of  a  process-oriented 
approach  rather  than  an  ordinary  input-output  information  approach  was  superior  for  training 
chemical  equipment  operation. 

It  was  easier  to  train  operators  who  were  given  flow  chart  information  depicting  the  process 
underling  the  chemical  plants  equipment  operation.  Myers  and  Fisk  (1987)  conducted  a  series  of 
experiments  designed  to  investigate  the  application  of  Shiffrin  and  Schneider’s  (1977)  theory  of 
automatic  and  controlled  processing.  One  major  hypothesis  of  this  study  that  was  supported  is 
that  high  performance  skills  develop  through  extended  practice  of  consistent  task  components. 

As  skill  develops  expert  performance  is  typically  controlled  through  automatic  processing. 

Smith  (1984)  was  also  interested  in  application  of  contemporary  cognitive  and  information 
processing  theory  to  complex  skill  training.  He  studied  the  effects  of  the  presentation  order  of 
cognitive  training  on  a  flight  task.  To  assess  order  effects,  Smith  presented  information  about 
the  cognitive  aspects  of  a  basic  flight  maneuver  at  various  times  in  the  flight  training  sequence. 

Four  groups  of  volunteer  subjects  were  used  in  the  study.  The  first  group  was  given  extensive 
cognitive  training  prior  to  undertaking  the  motor  task.  The  second  group  received  the  same 
extensive  cognitive  training  as  group  one  but  also  received  some  cognitive  training  during  motor 
skill  acquisition.  The  third  group  received  no  cognitive  training  prior  to  the  motor  test,  but 
received  sporadic  cognitive  instruction  during  motor  acquisition.  The  fourth  group,  serving  as 
control,  received  no  specific  cognitive  training,  but  was  only  given  motor  skill  instruction. 

The  results  of  this  study  showed  clearly,  that  group  one,  who  received  the  cognitive  training 
prior  to  undertaking  motor  skill  training,  performed  significantly  better  during  later  motor  skill 
practice  trials.  Groups  two  and  three,  using  sporadic  cognitive  training  during  motor  skill 
acquisition  showed  the  poorest  performance.  The  authors  concluded  that  providing  cognitive 
instruction  during  motor  skill  acquisition  trials  might  actually  impair  performance. 

Goettl  and  Gomez  (1995)  conducted  experiments  to  determine  the  effectiveness  of 
“observational  learning”  in  computer-based  simulation  learning.  Observational  learning  takes 
different  forms,  but  basically  the  term  refers  to  learning  that  takes  place  during  passive 
observation  of  task  performance,  such  as  that  provided  by  an  instructor’s  demonstration  of  a 
complex  task.  As  originally  outlined  by  Bandura  (1986)  learning  is  thought  to  depend  upon  such 
factors  as,  motivation,  attention,  and  the  individual  processing  of  critical  features  of  the  task 
during  the  period  of  observation.  A  behavioral  production  process  is  hypothesized  to  account  for 
the  transformation  of  cognitive  learning  attained  during  observation  to  performance  on  the 
perceptual-motor  task  components.  Carroll  and  Bandura  (1995)  further  stipulated  that  if  the 
spatial  and  temporal  features  of  the  task  can  be  easily  extracted  and  coded,  there  will  be  little 
need  for  overt  practice  (Goettl  and  Gomez,  1995,  p.  1335).  The  experimenters,  Goettl  and 
Gomez,  set  out  to  demonstrate  that  observational  learning  would  be  more  beneficial  in  situations 
that  can  be  symbolically  coded  than  for  tasks  that  cannot.  They  predicted  that  flight  simulation 
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tasks  that  have  a  substantial  cognitive  component  would  benefit  more  from  observational 
learning  that  would  tasks  that  are  substantially  perceptual-motor  in  content. 

The  results  of  a  series  of  interlocking  studies,  using  various  cognitive  and  perceptual-motor  tasks 
and  observational  learning  strategies,  were  consistent  with  other  research  findings  that  showed 
that  observational  learning  was  most  useful  for  tasks  with  high  cognitive  demands  than  those 
tasks  that  are  primarily  perceptual-motor.  These  findings  are  also  consistent  with  the  theories  of 
skill  acquisition  that  suggest  the  initial  stages  of  complex  perceptual-motor  skills  focus  on 
cognitive  or  declarative  knowledge  (Anderson,  1982;  1981). 

The  emphasis  on  observational  learning  made  by  Bandura  and  others  may  be  best  placed  on  the 
front  of  the  skill  acquisition'process  while  the  learner  is  still  attempting  to  understand  the 
elements  of  the  tasks  through  a  more  cognitive  problem-solving  approach. 


Prescriptive  Methods  of  Instruction  for  Perceptual-Motor  Learning 

As  was  discussed  in  an  earlier  section,  the  Component  Display  Theory  of  instruction  is  a  detailed 
methodology  for  teaching  cognitive  skills.  This  narrow  focus,  however,  limits  the  direct 
application  of  CDT  to  teaching  perceptual  motor  skills,  including  the  operation  of  complex 
equipment,  such  as  flying  an  airplane.  Ciavarelli  (1988,  1987)  recommended  that  CDT  be 
expanded  to  include  instructional  prescriptions  and  assessment  methods  for  complex  perceptual- 
motor  skill  training.  He  constructed  a  combined  task  classification  and  performance  matrix  and 
an  instructional  design  framework  for  the  complex  -perceptual-motor  learning  domain.  The 
extension  of  CDT  to  the  perceptual-motor  domain  was  accomplished  by  expanding  the  learning 
outcome  categories  originally  proposed  by  Merrill  (1983),  an  incorporating  appropriate 
instruction  presentation  strategies,  learning  aids,  and  assessment  methods.  A  basic  outline  of  this 
methodology  is  presented  below.  Clarification  and  additional  narrative  to  the  tables  and  bullets 
presented  below  will  be  provided  during  a  second  phase  of  the  SBIR  program. 
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Extended  CDT  for  Complex  Perceptual-Motor  Skills 


Task  Performance-  Perceptual  Content 


Spatial  Pattern 

Temporal  Pattern 

DETECT 

DISCRIMINATE 

RECOGNIZE 

Motor  Task  Classification 


Whole  Task 

Part  Task 

Self-Paced  or  Untimed 

Externally-Paced  or  Timed 

Primary  Instructional  Presentation  Strategies 

Primary  Presentations 


Whole 

Part 

DEMONSTRATION 

PRACTICE 

Secondary  Instructional  Presentation  Strategies 

•  Advance  Organizers 

•  Cue  Enhancement 

•  Verbal  Prompting 

•  Performance  feedback 

The  instruction  model  proposed,  like  CDT  for  the  cognitive  domain,  defines  specific  content  and 
task  performance  categories  as  defined  below: 

Perceptual  Content 

•  Signals:  single  elements  of  information  that  can  be  distinguished  from  surround  or 
background. 

•  Spatial:  arrangements  or  arrays  of  patterns  of  signals  or  objects 

•  Temporal:  correlated  time-event  sequences  or  sequence  patterns 
Performance  Levels  for  Perceptual  Skills 

•  Detection:  sensing  the  presence  or  absence  of  a  specific  signal  or  pattern 

•  Discrimination:  distinguishing  relevant  signals  and  patterns  from  non-relevant  ones 
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•  Recognition:  automatically  responding  to  specific  signal  categories  or  patterns 
Motor  Task  Content 

•  Discrete  task:  a  task  with  an  identifiable  beginning  and  end  point  (e.g.  hitting  a  baseball) 

•  Sequential  task:  a  series  of  discrete  motor  tasks  performed  in  a  specific  order  (e.g. 
shifting  a  five-speed  automobile  or  truck  transmission). 

•  Continuous  task:  requires  continuous  motion  from  start  to  end  of  task  performance,  and 
typically  is  guided  by  external  feedback  cues  (e.g.  manual  tracking  or  steering  a  vehicle) 

Performance  Levels  for  Motor  Skills 

•  Self-paced:  operator  can  control  the  speed  of  presentation  and  input  (and  work  at  own 
pace) 

•  Externally-paced:  the  speed  of  the  presentation  and  response  time  of  the  operator  is 
governed  by  the  system  or  environment 

The  extended  CDT  approach  described  here  presumes  that  the  learner  has  completed  the 
cognitive  segment  of  instruction  related  to  the  perceptual-motor  skill  using  prescriptive  methods 
of  instruction  described  previously.  The  cognitive  instruction  should  take  place  prior  to 
commencing  perceptual-motor  training.  The  learner  should  be  taught  the  nature  of  the  task  (facts 
about  operating  system,  performance  conditions  and  environment),  shown  examples  of 
information  flows,  salient  perceptual  cues,  and  given  some  instruction  about  associated  facts, 
concepts,  principles  and  procedures,  and  performance  requirements.  Some  examples  that 
illustrate  the  combined  Cognitive  and  Perceptual-Motor  CDT  instruction  are  provided  below: 


EXAMPLES  OF  LEARNING  OBJECTIVES  AND  INSTRUCTIONAL  METHODS 

(To  be  completed  for  various  Hover  Training  Tasks) 

1 .  Stationary  hover:  hover  3ft.  Behind  a  Maltese  cross,  in  alignment  with  the  runway,  at  a  skid 
height  of  3-5  ft. 

2.  Hover  taxi:  Taxi  down  the  centerline  of  the  runway  at  a  skid  height  of  3-5  ft  at  a  speed  not  to 
exceed  that  of  brisk  walk. 

3.  Hovering  turn:  Maintain  the  aircraft  over  a  fixed  point  on  the  runway  at  a  skid  height  of  3-5 
ft.;  perform  pedal  (yawing)  turn  to  the  right;  perform  90  degree  pedal  turn  to  the  left  to  return 
to  initial  heading. 

4.  Land  from  hover:  Smoothly  reduce  collective  pitch  to  land  from  a  stationary  hover, 
maintaining  alignment  and  position  over  the  ground 

5.  Takeoff  to  hover:  Smoothly  increase  collective  pitch  to  bring  the  aircraft  to  a  3-5  ft  hover 
maintaining  runway  alignment  and  position  over  the  ground  (Dohme,  1994,  p.l  17) 

Cognitive  Training  Segment 

1 .  Learning  objective  and  Performance  Level 

2.  Primary  Cognitive  Instructional  Presentations  (Rule,  Examples,  Practice,  and  Test) 

3.  Secondary  Cognitive  Instructional  Presentations  (Organizing  Templates,  Prompts  and 
Guidance) 
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Perceptual-Motor  Training  Segment 

1 .  Learning  objective  and  Performance  Level 

2.  Primary  Perceptual-Motor  Instructional  Presentations  (Instructor  Demonstrations,  Guided 
Practice,  After  Action  Feedback) 

3.  Secondary  Perceptual-Motor  Instructional  Presentations  (Cue  Accentuation,  or  Highlighting, 
Adjustment  of  Task  Difficulty  and/or  Information  Flow,  Instructor  Real-time  Assessment 
and  Guidance. 

EXPERT  TUTOR  SYSTEM 

An  Expert  Tutor  System  has  an  ability  to  infer  the  state  of  knowledge  and  intentions  of  the 
student  and  tailor  its  pedagogy,  instruction,  and  presentation  to  accommodate  the  user.  In  order 
to  do  this,  the  system  needs  to  create  and  maintain  a  student  model  of  the  current  user  as  depicted 
in  Figure  1.  An  expert  model  is  used  to  generate  inferences  about  the  student’s  state  of 
knowledge  represented  in  the  student  model.  A  presentation  module  would  use  the  results  of  this 
inference  “engine”  to  make  decisions  on  what  material,  or  task  performance  level  to  present  to 
the  learner,  and  how  best  to  present  that  material  or  task  simulation. 


Figure  1:  Basic  Intelligent  Tutoring  System 
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The  inference  engine  is  based  upon  the  use  of  artificial  intelligence,  or  in  this  application  referred 
to  as  an  expert  system.  An  expert  system  is  a  software  program  that  uses  artificial  intelligence 
techniques  for  reasoning  with  data  and  drawing  appropriate  inferences  about  instructional 
strategies  based  upon  a  students  profile  characteristics  and  performance  on  a  particular  task  or  set 
of  tasks.  Expert  systems  come  in  several  different  variations,  depending  on  the  structure  of  the 
knowledge  base  (rules,  frames,  or  semantic  networks),  the  method  of  reasoning  (forward 
chaining  or  backward  chaining),  ability  to  handle  uncertainty  (Bayesian  reasoning,  probabilistic 
models,  or  fuzzy  logic). 

Mulgund,  Asdigha,  and  Zacharias  (1995)  presented  a  refined  and  expanded  architecture  of  a 
general  intelligent  tutor  as  depicted  in  Figure  2,  below: 


Figure  2:  Basic  Intelligent  Tutoring  System  (Mulgind,  Asdigha  and  Zacharaias,  1995). 


Dohme  (1994)  became  interested  in  improving  training  for  beginning  US  Army  helicopter  pilots 
and  argued  for  the  application  of  sound  instructional  principles.  His  outline  of  key  learning 
processes  is  presented  below: 
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Summary  of  learning  processes  for  skill  development  (Dohme,  1994,  p.  51) 

1 .  An  indicator  on  which  the  activity-relevant  indication  appears  (stimulus)  (e.g., 
instruments,  warning  lights,  and  horns.) 

2.  A  cue,  or  sign  that  calls  for  a  response  (decision  to  make  corrective  action)  (e.g. 
checklist,  instrument  reading  different  from  expected,  flashing  light,  a  special  sound). 

3.  A  control  object  to  be  activated  (e.g.  aircraft  yoke,  throttles,  rudder  pedals). 

4.  The  activation  or  manipulation  to  be  made  (actual  behavior  sequence  in  executing  the 
selected  motor  action)  (e.g.  push  forward,  pull  back,  turn  clockwise). 

5.  The  indication  of  response  adequacy  (e.g.  instrument  reading  normal,  glide  -slope 
indicator  or  showing  on  glide  slope,  flashing  light  goes  off). 

Dohme  suggested  that  an  Automated  Helicopter  Hover  Trainer  be  developed  based  upon  our 
understanding  of  how  best  to  teach  hover  tasks.  He  made  the  following  key  points  in  his  analysis 
(Dohme,  1994,  p.  115). 

Key  points: 

•  Hovering  is  one  of  the  key  skills  learned  by  ab  initio  trainees. 

•  Stationary  hover,  hover  taxi,  hovering  turns,  and  takeoff  and  land  from  hover  must  be 
mastered  before  a  helicopter  pilot  can  solo 

•  Hovering  requires  the  coordinated  use  of  helicopter  flight  controls 

•  Pilot  must  learn  to  ‘overcome  the  interactions  built  into  the  aircraft 

•  Learning  to  hover  as  been  traditional  “monkey  see  monkey  do”  approach,  in  which  instructor 
pilot  demonstrates  a  maneuver  and  then  the  trainee  performs  the  maneuver  while  the  IP 
monitors  and  guides  performance. 

•  Pilot  must  learnt  o  maintain  the  aerodynamic  balance  required  for  stable  flight. 

•  This  requires  the  pilot  to  attend  to  specific  cues  and  to  make  small  but  accurate  control  inputs 
to  maintain  a  constant  position  over  the  ground. 

Dohme  (1994)  stated  that,  “A  simulator-based  trainer  is  envisioned  that  would  continuously 
review  trainee  performance  and  adaptively  augment  control  inputs  such  that  the  demand 
characteristics  of  the  simulator  would  accommodate  the  trainee’s  ability  to  successfully  hover  (p. 
116).”  A  prototype  was  developed  and  tested  in  a  series  of  experimental  training  transfer 
studies.  The  automated  hover  trainer  used  a  mathematical  model  that  compared  trainee 
performance  against  “expert  performance  norms”  that  were  derived  from  analysis  of  maneuvers 
performed  by  highly  experienced  pilots.  The  computer  program  ‘adapts”  to  the  learners 
performance  by  reducing  the  control  augmentation  as  the  performance  improves  up  to  the 
standard  (i.e.  the  student  is  performing  to  the  level  defined  in  the  unaugmented  helicopter 
aerodynamic  model). 
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Intelligent  Flight  Trainer 

This  SBIR  project  completed  by  Mulgund,  Asdigha  &  Zacharias  (1995)  set  out  to  integrate 
intelligent  tutoring  with  PC-based  flight  simulation.  IP  domain  knowledge  was  incorporated  into 
an  expert  system  with  the  intent  of  providing  intelligent  corrective  feedback  to  the  trainee.  The 
feedback  was  represented  by  verbal  comments  using  a  voice  synthesizer.  Both  procedural 
cueing  on  “proper  maneuver  execution”,  and  comments  regarding  “manual  task  performance” 
were  provided. 

Some  of  the  key  functional  characteristics  of  the  IFT  baseline  system  include  (p.  1) 

•  A  tutorial  function  to  remind  the  student  how  to  implement  good  perceptual  or 
control  strategies  (e.g.  “Remember  to  apply  left  pedal  as  you  raise  the  collective.”). 

•  A  performance  monitoring  function  for  performance  advisory  messages  (e.g.,  “You 
are  too  high.”). 

•  A  control  activity  monitoring  function  to  provide  feedback  on  the  student’s  control 
usage  (e.g.  “You  are  thrashing  the  collective.”). 

•  An  advisory  function  that  makes  explicit  the  suggested  control  strategy  or  corrective 
maneuvering  (e.g.  “Slow  down  using  aft  cyclic.”). 

System  design  software  components  included,  a  rule  base,  a  message  database,  and  adaptive 
control  logic  that  governed  the  hover  maneuvers.  The  IFT  determines  stability  augmentation 
needed  on  the  basis  of  performance  monitoring,  and  then  sends  inputs  to  the  host  simulator  for 
controlling  helicopter  flight  dynamics.  Cockpit  displays  are  updated  as  soon  as  flight  dynamics 
data  are  available  to  the  simulator  host. 

Mulgund,  Asdigha,  and  Zacharias  (1995)  summarize  a  Phase  II  SBIR  effort  for  development  and 
validation  of  an  Intelligent  Flight  Trainer  (IFT).  The  IFT  is  a  PC-simulator  system  used  to  teach 
beginning  rotary-wing  students  basic  hover  maneuvers.  This  trainer  includes  an  expert  system 
representing  the  instructor  pilot  (IP)  that  presents  feedback  to  students  during  practice  of  hover 
maneuvers  using  synthetic  voice.  The  expert  system  shell  drives  a  variable  stability 
augmentation  algorithm  that  adjusts  the  difficulty  of  helicopter  motion  control,  to  make  it  easier 
for  the  novice  to  maintain  stability  and  control  of  the  vehicle.  The  IFT  was  completed  and  major 
functional  performance  areas  were  demonstrated,  including  adaptive  aiding  that  varied  task 
difficulty,  IP  performance  diagnosis  and  generation  of  student  advice,  and  use  of  synthetic  voice 
feedback  system.  The  IFT  study  report  (Mulgund,  Asdigha,  and  Zacharias,  1995),  recommended 
that  follow-on  efforts  be  undertaken  as  follows  (p.  v): 

1 .  Conduct  a  formal  validation  and  transfer  experiment. 

2.  Transition  to  the  TH-67  helicopter. 

3.  Develop  training  modules  to  include  other  flight  tasks  (level  flight,  climb,  turns, 
descents,  etc. 

4.  Install  all  system  components  on  a  single  PC  (rather  than  linked  PC’s  as  is  current 
baseline  system  is  configured  now). 
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Other  recommendations  include  the  inclusion  of  graphic  augmentation  and  feedback,  including 
the  possibility  of  using  a  “highway  in  the  sky”  display,  and  using  visual  display  cue 
augmentations  such  as  highlighting  or  pointing  to  important  display  data  elements. 

The  functional  architecture  of  the  IFT  is  shown  in  Figure  3,  below. 

Figure  3:  Baseline  one:  Intelligent  Flight  Trainer  (Mulgund,  Asdigha  and  Zacharaias, 
1995). 


The  simulator  host  models  the  6-degree  of  freedom  dynamics  of  a  Bell  UH-1  (Huey)  helicopter 
developed  at  the  University  of  Alabama  Flight  Dynamics  Laboratory  (Mulgund,  Asdigha,  & 
Zacharias,  1995,  p.  14).  Two  major  computational  models  are  used  in  the  IFT. 
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1 .  Expert  System  Advisor  that  uses  instructor  pilot  knowledge  to  produce  feedback  on 
manual  flight  performance  and  corrective  piloting  techniques. 

2.  Adaptive  Helper  that  provides  vehicle  dynamics  stability  augmentation  to  adjust 
control  task  difficulty  to  match  a  particular  student’s  skill  level. 

The  domain  knowledge  uses  a  CLIPS  rulebase  developed  by  NASA  to  help  model  human 
knowledge  or  skill  (Giarratano,  1983).  A  complete  description  and  mathematical  model 
definition  is  presented  in  Mulgund,  et  al.  and  will  not  be  repeated  in  any  great  detail  here. 

Briefly,  the  Expert  System  Advisor  provides  verbal  feedback  to  the  student  using  a  synthetic 
voice  system.  Feedback  oontent  and  format  varies  as  follows  (p.  15): 

•  A  tutorial  function  to  remind  student  how  to  implement  good  perceptual  or  control  strategies 
(e.g.  “remember  to  apply  left  pedal  as  you  raise  the  collective.”) 

•  A  performance  monitoring  function  for  performance  advisory  messages  (e.g.,  “You  are  too 
high.”). 

•  A  control  activity  monitoring  function  to  provide  feedback  on  the  student’s  control  usage 
(e.g.,  “You  are  thrashing  the  collective.”). 

•  An  advisory  function  that  makes  explicit  the  suggested  control  strategy  or  corrective 
maneuvering  (e.g.  “Slow  down  using  aft  cyclic.”). 

The  expert  system  provides  appropriate  feedback  based  upon  student  performance  on  the  task, 
and  informs  the  student  of  any  changes  is  in  help  function  adjustments. 

Again  briefly,  the  Adaptive  Helper  generates  “inner-loop”  stability  augmentation  to  facilitate 
student  control  of  the  helicopter  and  assist  in  performing  a  particular  maneuver.  The  strategy 
used  by  the  adaptive  helper  is  based  on  modeling  the  student  pilot  and  the  expert  pilot  using 
methods  developed  by  Kleinman,  Baron,  &  Levison,  (1970)  and  referred  to  as  the  Optimal 
Control  Model. 

The  system  supports  aided  training  on  such  tasks  as:  fixed  hover  (student  maintains  the  helicopter 
in  a  fixed  position  over  the  runway),  hover  taxi  (student  begins  at  a  fixed  point  and  hovers  down 
runway  centerline  for  a  defined  distance),  hover  turn  (the  student  makes  alternating  right  and  left 
turns  while  maintaining  fixed  coordinate  limits),  traffic  pattern  (the  student  flies  a  rectangular 
flight  pattern  near  the  runway),  accelerate  to  lift  off  position  (student  accelerates  down  runway 
centerline  and  maintains  lateral  and  altitude  flight  parameters  to  lift  of  point),  and  for  takeoff  and 
landing  from  a  hover.  More  complete  information  on  mathematical  models,  hardware  and 
software  implementation  of  the  models,  control  programs,  and  simulator  is  presented  in 
Mulgund,  Asdigha,  &  Zacharias  (1995). 

Monterey  Technologies,  Inc 

Skill  training  requires  both  instruction  and  the  opportunity  to  practice  the  task.  One  method  of 
instruction  would  be  to  provide  “show  and  tell”  guidance  initiated  by  an  external  agent  based  on 
the  presumed  or  perceived  needs  of  the  student.  Instruction  facilitates  learning  but  is  not  an 
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essential  element  of  the  learning  itself.  Feedback  is  the  immediate  and  direct  sensory  and 
perceptual  consequences  of  the  student’s  actions  (or  inaction)  and  is  an  essential  element  of  the 
learning  process.  Skill  acquisition  depends  on  the  feedback  gleaned  from  frequent  and  consistent 
pairings  of  actions  with  external  event  outcomes.  An  after-performance  critique  is  often  referred 
to  as  feedback,  but  is  really  a  performance  history  based  upon  recently  completed  behavior.  This 
critique  usually  is  intended  to  instigate  a  modification  of  the  student’s  behavior  during  the  next 
training  exercise.  Instruction  during  student  performance  is  an  abbreviated,  short  cycle  of 
instructional  critique  and  not  a  form  of  feedback.  Figure  4  shows  a  schematic  model  of  a  typical 
training  process.  Note  that  the  instructor  must  both  assess  the  skill  level  of  the  student  and  make 
recommend  any  changes  to  either  the  feedback  received  by  the  student  of  the  instruction  given  to 
the  student. 

Figure  4:  Alternate  Baseline  Intelligent  Tutor.  Initial  baseline  presented  by  Monterey 
Technologies,  Inc.  (SBIR  Proposal,  1999). 
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Figure  5  presents  an  evolved  version  of  Figure  4  with  the  live  instructor  replaced  by  a  model  and 
a  model  of  the  student  added  to  allow  interpretation  of  the  performance  and  learning  processes  of 
the  student.  The  key  feature  of  this  tutoring  system  is  its  dependence  on  behaviorally  based 
models  of  learning,  performance  and  instruction  that  are  open  to  scrutiny  and  modification. 


SBIR  DATA  RIGHTS  Contract  No.  DASW01-00-M-4084.  Contractor  Name:  Monterey  Technologies,  Inc.,  Address:  26400  Silver  Cloud  Ct.,  #103, 
Monterey,  CA  93940.  Expiration  of  SBIR  Data  Rights  Period:  July  1 . 2005.  The  Government’s  rights  to  use,  modify,  reproduce,  release,  perform, 
display,  or  disclose  technical  data  or  computer  software  marked  with  this  legend  are  restricted  during  the  period  shown  as  provided  in  paragraph 
(b)(4)  of  the  Rights  in  Noncommercial  Technical  Data  and  Computer  Software- Small  Business  Innovation  Research  (SBIR)  Program  clause 
contained  in  the  above  identified  contract.  No  restrictions  apply  after  the  expiration  date  shown  above.  Any  reproduction  of  technical  data, 
computer  software,  or  portions  thereof  marked  with  this  legend  must  also  reproduce  the  markings. 


Appendix  1-28 


Monterey  Technologies,  Inc, 


Contract  No.  DASW01-00-M-4084 


SB1R  Phase  1  Final  Report 


Figure  5:  Baseline  3.  Intelligent  Tutor  (Chappell  and  Mitchell,  1997). 
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Sleeman  and  Brown  (1982)  raised  several  important  issues  regarding  the  use  of  intelligent  tutors, 
following  a  review  of  artificial  intelligence  (AI)  applications  to  instruction,  and  early  versions  of 
tutoring  systems.  The  limitations  outlined  by  these  authors  is  worth  keeping  in  mind  today,  in 
spite  of  the  many  improvements  in  AI,  and  computing  technology  available  now. 

1 .  The  instructional  material  produced  in  response  to  a  student’s  query  or  mistake  is 
often  at  the  wrong  level  of  detail,  as  the  system  assumes  too  much  or  too  little  student 
knowledge. 

2.  The  system  assumes  a  particular  conceptualization  of  the  domain,  thereby  coercing 
the  student’s  performance  into  its  own  conceptual  framework. 

3.  The  tutoring  and  critiquing  strategies  used  by  these  systems  are  excessively  ad  hoc 
reflecting  unprincipled  intuitions  about  how  to  control  their  behavior. 

4.  User  interaction  is  still  too  restrictive,  and  limits  the  student  to  work  with  the  ability 
of  the  tutors  diagnostic  mechanisms  (Sleeman  &  Brown,  1982,  p.3). 

One  of  the  primary  limitations  of  AI  applications  to  instruction  mentioned  by  Sleeman  and 
Brown,  and  still  operative  today,  is  the  lack  of  a  consistent  set  of  learning  principles  and  precise 
domain  knowledge  on  which  to  construct  rule  based  systems  that  fully  represent  the  student  and 
the  instructor. 


Knowledge  base  for  Perceptual-motor  Learning 

Information  extracted  from  the  literature  in  perceptual -motor  learning  and  some  findings  from 
the  cognitive  learning  literature  are  summarized  in  Attachment  B.  It  is  intended  that  the 
summary  provided  in  Attachment  B  serve  as  a  point  of  departure  for  generating  a  rulebase  of 
pedagogy  for  teaching  complex  perceptual  motor  skills,  and  is  particularly  applicable  to  flight 
instruction. 
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ATTACHMENT  A 

SYNOPSES  OF  SELECTED  REFERENCES 
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REFERENCE  SYNOPSIS 

Ackerman,  P.L.  (1983).  A  theory  for  predicting  ability/skill  relations:  An  approach  from 
automatic  and  controlled  processing.  In,  Proceedings  of  Human  Factors  Society — 27th  Annual 
Meeting. 

ABSTRACT:  Studies  related  to  perceptual-motor  learning  are  reviewed  and  discussed.  The 
authors  propose  a  general  theory  of  the  relationship  between  abilities  and  aptitudes.  The  theory  is 
based  upon  an  understanding  of  controlled  and  automatic  processing  and  various  cognitive 
theories. 

KEY  TOPICS  AND  ISSUES: 

1 .  The  general  theory  proposed  by  Ackerman,  in  part,  accounts  for  the  common  finding 
regarding  changes  in  the  inter-correlation  between  early  learning  trials  and  late  learning  trial 
performance  scores. 

•  Patterns  of  correlation  coefficients  change  substantially  over  the  course  of  training. 

•  Typically  measures  of  cognitive  ability  correlate  highly  with  initial  training,  but  then 
decline  in  magnitude  as  training  on  an  operational  task  continues. 

•  One  interpretation  of  this  correlation  pattern  is  that  cognitive  components  of  the  task  are 
more  influential  during  early  skill  learning,  and  that  perceptual-motor  components  have 
greater  influence  as  learning  progresses. 

2.  The  theory  of  controlled  and  automatic  processing  proposed  by  Schneider  and  Shiffrin 
(1977)  typically  observes  changes  in  operator  performance  characterized  by  “effortful,  slow, 
and  error  prone”  performance  during  early  training  periods,  and  progressively  less  effort, 
improved  accuracy,  resistance  to  distraction  and  a  greater  level  of  automaticity  in 
performance,  as  the  operator  becomes  more  proficient. 

•  Shiffrin  and  Schneider  refer  to  these  two  forms  of  processing  as  controlled  processing 
(CP)  and  automatic  processing  (AP).  Controlled  processing  is  under  conscious 
control,  and  easily  modified  during  performance.  In  automatic  processing,  task 
performance  is  largely  unconscious  and  unfolds  much  like  a  stored  computer 
program,  and  hence  is  less  modifiable  during  task  performance. 

•  Later  studies  by  Fisk  and  Schneider  (1981)  indicate  that  transformation  from 
controlled  to  automatic  performance  described  may  only  be  true  for  tasks  that  have 
consistent  task  elements  (in  which  the  operator  responds  to  predictable  relationships 
between  perceptual  cues  and  responses). 

•  The  consistent  components  of  a  complex  skill  stabilize,  during  early  during  training 
and  the  patterns  of  correlation  due  not  change  appreciably  over  the  course  of  skill 
development. 
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•  Skill  automaticity  reflects  the  learners’  ability  to  acquire  essential  perceptual-motor 
task  components  based  in  internalized  relationships  between  consistent  elements  of 
tasks  and  relevant  perceptual  cues. 

APPLICATION  OF  FINDINGS 

•  Instructional  designers  should  attempt  to  identify  cognitive  and  perceptual-motor 
components  of  a  complex  skill  in  order  to  develop  effective  instructional  strategies. 

•  In  accordance  with  the  findings  summarized  here,  it  may  be  most  beneficial  to  offer 
cognitive  approaches  early  in  training,  and  then  shift  to  perceptual-motor  aiding  later  in  the 
training  period. 

•  Instructional  designers  should  identify  context  (environmental)  cues  that  are  the  most 
powerful  in  capturing  attention  and  initiating  correct  or  incorrect  responses  during  later 
periods  of  training. 

RELATED  REFERENCES: 

Fisk,  A.D.,  &  Schneider,  W.  (1981).  Control  and  automatic  processing  during  tasks  requiring 

sustained  attention:  A  new  approach  to  vigilance.  Human  Factors.  23.  737-750. 

Fleishman,  E.A.  (1972).  On  the  relation  between  abilities,  learning  and  performance.  American 

Psychologist.  27.  1017-1032. 
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REFERENCE  SYNOPSIS 

Fisk,  A.D.,  Lee,  M.D.,  &  Rogers,  W.A.  (1991)  Recombination  of  automatic  processing 
components:  The  effects  of  transfer,  reversal,  and  conflict  situations.  Human  Factors.  33.  (3), 
267-280. 

ABSTRACT:  This  paper  discusses  issues  related  to  the  automation  of  skilled  performance.  The 
study  reports  findings  of  experiments  investigating  the  effects  of  compatible  and  incompatible 
automatic  process  on  performance.  Fisk  and  his  associates  amplified  on  earlier  work  of  Fitts 
(1962,  1967)  regarding  the  automation  of  skill  as  a  performer  moves  from  novice  to  expert. 

Most  skill  performance  progressively  improves  in  speed  and  precision,  as  well  as  a  reduction  in 
attention  resources  required' for  a  given  task.  A  skilled  performer  appears  to  perform  complex 
tasks  quite  effortlessly  after  substantial  practice.  Fitts  and  his  colleagues  have  identified  some 
specific  relationships  underlying  skilled  performance  that  may  be  useful  in  establishing 
principles  of  learning  and  performance  enhancement.  It  is  strongly  implied  that  the  research 
foundation  is  well  enough  established  to  recommend  specific  "  limits  and  guidelines"  for 
training. 

KEY  TOPICS  AND  ISSUES: 

Selected  guidelines  recommended  include  (p.  268-270): 

•  Performance  improvements  (with  practice)  will  only  occur  for  consistent  tasks. 

•  The  type  and  number  of  inconsistent  task  elements  limit  performance  improvement. 

•  The  degree  of  consistency  among  stimuli,  rules  and  context  are  factors  to  consider  in  part- 
task  learning  strategies. 

•  Context  is  an  important  element  affecting  skilled  performance,  in  part,  because  contextual 
cues  may  trigger  appropriate  or  inappropriate  automatic  performance  processes. 

APPLICATION  OF  FINDINGS 

•  Instructional  designers  should  attempt  to  identify  and  consider  apparent  consistencies  in  the 
task  structure  of  skills  to  be  trained. 

•  Instructional  designers  should  identify  context  (environmental)  cues  that  are  the  most 
powerful  in  capturing  attention  and  initiating  correct  or  incorrect  responses  during 
acquisition  of  skilled  performance. 

RELATED  REFERENCES: 

Fisk,  A.D.,  and  Gallini,  J.K.  (1989).  Training  consistent  components  of  tasks:  Developing  an 
instructional  system  based  on  automatic/controlled  processing  principles.  Human  Factors.  31, 
453-63. 

Logan,  G.D.  (1988).  Toward  an  instance  theory  of  automatization.  Psychological  Review.  95, 
492-527. _ 
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REFERENCE  SYNOPSIS 

Fitts,  P.M.  (1962).  Factors  in  complex  skill  training,  In,  R.Glaser  (Ed.)  Training  research  and 
education.  PA:  University  of  Pittsburgh  Press. 

ABSTRACT:  This  paper  summarizes  early  work  by  Paul  Fitts  in  the  area  of  skill  acquisition. 
Skilled  performance  is  defined  as  having  three  key  characteristics,  spatial-temporal  patterning, 
continuous  interactions  between  input,  output  and  feedback  processes,  and  learning.  The  basic 
rudiments  of  perceptual-motor  skill,  including  basic  task  taxonomy  and  characteristics  of 
environmental  and  internal  cues  are  described.  The  author  discusses  the  learning  of  complex 
skills  in  terms  of  "continuous"  learning  phases,  including  Cognitive,  Fixation,  and  Autonomous. 
Paul  Fitts  was  an  early  pioneer  in  skill  research,  and  much  of  the  work  related  to  skill 
acquisition,  retention,  and  measurement  of  perceptual  motor  performance  is  based  on  his  work. 

KEY  TOPICS  AND  ISSUES: 

1 .  Components  of  a  complex  skill  outlined  by  Fitts  include: 

•  Cognitive  —  understanding  the  structure  or  nature  of  the  task. 

•  Perceptual  —  learning  what  to  pay  attention  to,  what  to  look  for  (salient  cue 
recognition  and  discrimination) 

•  Physical  Coordination  —  integration  of  perceptual  and  motor  activities  (timing  of 
movement  patterns. 

•  Tension  Relaxation  -  greater  relaxation  a  smoothness  and  precision  with  less  effort 
expended. 

2.  Phases  of  skill  development 

•  Cognitive  —  conscious  analysis  and  verbalization  of  tasks  and  cues. 

•  Fixative  or  associative  —  correct  perceptual-motor  patterns  emerge. 

•  Autonomous  --  the  automation  of  skilled  performance,  with  improved  speed, 
smoothness  and  accuracy.  At  this  stage  of  learning  there  is  less  conscious  control  and 
more  resistance  to  distraction  and  a  shift  from  external  cues  to  internal 
(proprioceptive)  cues. 

3.  Task  Classification  must  consider  the  task  and  conditions  of  performance  (body  position  and 
motion  as  well  as  the  type  and  motion  of  environmental  perceptual  targets  or  cues).  The 
most  complex  tasks  are  those  in  which  the  performer  is  in  motion,  and  the  environment  is 
moving  or  changing  rapidly  as  well.  The  following  taxonomy  presents  this  particular  view: 
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COMPLEX  PERCEPTUAL-MOTOR  SKILL  TAXONOMY 
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4.  In  complex  perceptual-motor  tasks,  like  flying,  the  performer  must,  keep  monitor  many 
separate  sources  of  information,  and  sort  out  the  effects  of  the  changing  environment,  including 
those  effects  produced  by  the  performers  own  actions. 

APPLICATION  OF  FINDINGS 


1 .  Characterizations  of  skill  taxonomy  and  learning  imply  various  levels  of  complexity,  and 
progressive  levels  of  skilled  performance. 

2.  In  practice,  good  instructors  understand  that  teaching  complex  skills  requires  different  types 
and  levels  of  information  and  feedback  to  the  student.  So  common  strategies  taken  by  instructors 
based  upon  experience  are  supported  by  concepts  proposed  by  Fitts  are: 

•  Provide  students  with  tutorial  information  regarding  the  task  structure  and  salient 
cues  to  pay  attention  to. 

•  Demonstrate  correct  performance  to  student,  showing  correct  response  sequence  and 
timing. 

•  Reinforce  attention  to  salient  cues  during  practice  sessions,  and  provide  feedback  of 
performance  results. 

•  As  skill  becomes  more  automated,  provide  less  coaching  and  verbal  support  during 
task  performance. 

RELATED  REFERENCES: 


Fitts,  P.M.  and  Posner,  M.I.  (1967).  Human  performance.  Belmont,  CA:  Brooks-Cole 

Merrill,  M.D.  (1971).  Psychomotor  taxonomies,  classifications,  and  instructional  theory,  In, 
R.N.  Singer  (Ed.),  The  Psvchomotor  Domain.  NY :  Academic  Press. 
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REFERENCE  SYNOPSIS 

Lintern,  G.  (1991).  An  informational  perspective  on  skill  transfer  in  human-machine  systems. 
Human  Factors.  33.  (3),  251-266. 

ABSTRACT:  This  paper  discusses  skilled  performance  and  transfer  from  the  perspective  of 
perceptual  learning  theory.  The  basis  for  learning  and  transfer  of  complex  skills  is  considered  to 
be  a  performer's  attention  to  specific  perceptual  patterns  or  invariants  in  the  stimulus 
environment,  and  their  relationship  to  the  task.  The  key  theoretical  formulation  proposed  is  that 
both  learning  and  transfer  of  complex  skills  is  based  upon  progressive  detection  and 
discrimination  of  critical  perceptual  cues,  stimulus  patterns,  and  important  environmental 
features  that  serve  as  external  feedback  for  skilled  performance.  These  perceptual  invariants  and 
their  positive  or  negative  correspondence  among  different  tasks  may  account  for  transfer 
differential  learning  transfer  effects. 

KEY  TOPICS  AND  ISSUES: 

1 .  Transfer  of  a  particular  skill  depends  upon  the  nature  of  the  task,  the  control  characteristics  of 
the  operating  device,  and  the  particular  design  of  the  human-machine  interface. 

•  Sometimes  skill  transfer  is  negative,  that  is  practice  on  the  first  task  interferes  with 
performance  or  delays  learning  on  a  second  task. 

•  Sometimes  previous  experience  on  a  task  improves  performance  on  a  second  task  or 
facilitates  learning  a  second  task  ( positive  verses  negative  learning  transfer) 

2.  The  similarity  between  tasks  (correspondence  between  stimulus  environments  and/or 
response  patterns)  and  task-environment  fidelity  from  a  simulated  activity  to  an  operational 
environment  are  typically  proposed  to  account  for  positive  and  negative  transfer  effects.  But  it 
has  often  been  shown  that  positive  learning  transfer  can  occur  between  tasks  that  are  quite 
different  in  composition  or  are  performed  in  different  environments. 

•  In  some  studies,  "deliberate  departures"  from  similarity  or  fidelity  resulted  in  positive 
transfer  effects.  Lintern  uses  the  example  of  training  a  pilot  to  land  in  a  cross-wind, 
and  discusses  findings  from  Lintern,  Roscoe,  and  Sivier,  (1990)  which  showed  that 
pilots  trained  without  a  cross-wind  conditions  actually  did  better  on  a  transfer  task 
that  included  cross-wind  landings. 

•  Lintern  concludes  that  transfer  effects  can  not  be  fully  explained  using  "similar 
elements"  theory,  or  level  of  simulation  fidelity. 

3.  Lintern  proposes  that  high-performance  skills,  in  particular  manual  control  skills,  are  learned 
through  a  process  of  identifying  critical  features  of  the  stimulus  environment  that  distinguish 
correct  from  incorrect  performance  on  a  particular  task. 

4.  There  is  a,  "lawful  relationship  between  patterns  of  stimulation  and  properties  of  the  task." 
Some  examples  of  so-called  invariants,  or  stimulus  patterns  that  pilots  use  are: 

•  Motion  cues  derived  from  "optical  flow",  or  the  relative  rate  at  which  a  visual  scene 
changes  with  respect  to  an  aircraft's  speed  and  turn-rate. 

•  Size-Distance  cues  derived  from  perception  of  the  ratio  of  runway  width  to  runway 
_ length.  This  is  a  powerful  cue  relationship  that  enables  pilots  to  fly  a  glideslope,  and 
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may  sometimes  lead  to  common  misperceptions  of  slant  range  if  this  ratio  is 
unexpectedly  large  or  small. 

4.  Learning  is  a  consequence  of  progressive  understanding  of  the  perceptual  cues  and  their 
relationship  to  task  performance,  such  that  the  learner  progressively  improves  in  his/her 
discrimination  and  extraction  of  critical  information  from  the  environment  needed  for  "expert" 
performance. 

APPLICATION  OF  FINDINGS 

1 .  Learning  and  transfer  may  be  improved  by  helping  learner  to  identify  critical  features  of  the 
stimulus  environment  that  are  essential  for  task  performance. 

2.  Helping  learner  to  identify  information  and  situations  leading  to  error  or  degraded 
performance  may  improve  learning  and  transfer. 

3.  Simulation  devices  can  include  instructional  features  that  enable  the  instructor  to  accentuate 
or  highlight  specific  features  of  the  task  and  environment. 

4.  Learning  and  transfer  may  be  improved  by  having' an  "expert"  (instructor)  demonstrate 
correct  and  incorrect  task  performance,  while  highlighting  associated  perceptual  cues  related 
to  task  performance. 

RELATED  REFERENCES: 

Lintem,  G.,  Roscoe,  S.N.,  &  Sivier,  J.  (1990).  Display  principles,  control  dynamics,  and 
environmental  factors  in  pilot  performance  and  transfer  of  training.  Human  Factors,  32,  299-317. 

Wightman,  D.C.,  &  Sistrunk,  F.  (1987).  Part-task  training  strategies  in  simulated  carrier  landing 
final  approach  training.  Human  Factors,  29,  245-254. 
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REFERENCE  SYNOPSIS 

Myers,  G.L.,  &  Fisk,  A.D.  (1987).  Training  consistent  task  components:  Application  of 
automatic  and  controlled  processing  theory  to  industrial  task  training.  Human  Factors.  29,  (3). 
255-268. 

ABSTRACT:  Two  experiments  were  conducted  to  examine  the  generality  of 
automatic/controlled  processing  training  principles  to  rich,  complex  tasks.  In  both  experiments, 
subjects’  tasks  were  modeled  after  a  job  function  performed  by  the  telecommunications  industry. 
These  tasks  required  subjects  to  process  conjunctions  of  information.  Large  quantitative  and 
qualitative  differences  were  found  between  the  consistently  and  variably  mapped  training 
conditions.  The  need  for  determining  trainable  consistent  components  of  complex  tasks  is 
discussed  (Authors,  p.  255) 

KEY  TOPICS  AND  ISSUES 

1 .  Most  instructional  design  theories  do  not  fully  consider  the  changes  that  take  place  during 
skill  acquisition,  and  do  not  provide  guidelines  regarding  proper  instructional  strategies  for 
different  phases  of  skill  development. 

2.  Schneider  and  Shiffrin  (1977)  outlined  a  theory  of  skill  automation  in  their  descriptions  of 
controlled  and  automatic  performance.  Controlled  processing  is  a  relatively  slow,  typically 
serial,  “effortful”,  capacity  limited,  and  subject-regulated  mode.  Whereas  automatic 
processing  is  faster,  parallel,  not  typically  under  (conscious)  subject  control,  and  not  as 
resource  limited. 

3.  Controlled  processing  is  usually  applied  under  conditions  that  are  novel,  or  during  early 
learning  phases,  of  skill  development.  Automatic  processing  typically  relates  to  the 
consistent  task  components  that  represent  invariant  responses  to  particular  environmental 
cues  or  stimuli. 

4.  Practice  on  the  consistent  task  components  accounts  for  the  eventual  automation  of  the  skill 
following  extensive  practice. 

APPLICATION  OF  RESULTS 

The  consistency  principle  is  most  relevant  to  development  of  instructional  strategies  for  complex 
skills. 

•  One  instructional  strategy  would  be  to  isolate  specific  consistent  task  elements  of  a  skill 
and  allow  the  learner  to  practice  (part-task)  such  that  the  trainee  receives  numerous 
correct  executions  of  the  consistent  tasks  components  of  a  specific  skill. 

•  The  above-summarized  study  supports  this  framework  with  the  experimental 
demonstration  that  some  rather  complex  tasks  can  be  effectively  trained  in  this  manner. 


RELATED  REFERENCES 
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Schneider,  W.,  &  Shiffrin,  R.M.  (1977).  Controlled  and  automatic  human  information  processing 

I.  Detection,  search,  attention.  Psychological  Review,  84,  1-66. 

Shiffrin,  R.M.,  &  Schneider,  W.  (1977).  Controlled  and  automatic  human  information  processing 

II.  Perceptual  learning,  automatic  attending,  and  general  theory.  Psychological  Review.  84,  127- 
190. 
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REFERENCE  SYNOPSIS 

Schneider,  W.  (1985).  Training  high-performance  skills:  Fallacies  and  Guidelines.  Human 
Factors,  27,  (3).  285-300. 

ABSTRACT:  The  authors  make  the  point  that  all  trainees  do  not  reach  the  levels  of  proficiency 
desired  in  many  systematic  approaches  to  training,  because  many  training  programs  are  based 
upon  false  assumptions  about  human  learning.  This  paper  reviews  and  discusses  six  of  the 
common  training  fallacies. 

KEY  TOPICS  AND  ISSUES: 

1 .  Most  learning  studies  are  of  too  short  a  duration  to  draw  broad  generalizations  about  learning, 
and  training  effectiveness. 

2.  High-performance  skills  are  characterized  by  three  things: 

•  The  length  of  time  to  become  competent  and  proficient  at  the  task. 

•  A  typically  high  failure  and  non-completion  rate  for  training. 

•  There  is  a  considerable  performance  difference  between  a  novice  and  an  expert. 

3.  Novices  appear  “overtaxed”  and  are  easily  distracted  during  performance,  whereas  the  expert 
appears  to  perform  smoothly  and  without  effort,  and  is  not  easily  distracted. 

4.  As  one  moves  progressively,  from  novice  to  expert,  the  performance  changes  (in  terms  of 
competency  exhibited  and  very  likely  in  terms  of  the  information  processing  and  control  tasks 
performed). 

5.  The  expert  makes  decisions  much  more  rapidly  based  upon  experience,  and  without  much 
“thinking”  or  deliberation. 

6.  Since  the  composition  of  the  skill  itself  appears  to  change  over  time,  then  it  would  seem  that 
the  conditions  of  learning  and  the  training  strategy  must  change  as  well. 

7.  Most  training  programs  to  not  take  the  changes  in  skill  composition,  or  a  learners  progression 
from  novice  to  expert  into  full  consideration  in  arriving  at  the  best  instructional  strategies  for  a 
learner’s  progression  in  skill  development. 

8.  Some  of  the  more  common  fallacies  of  learning  are  as  follows  (pp.  287-89): 

•  Fallacy  1:  Practice  makes  perfect 

•  Fallacy  2:  Training  of  the  total  skill 

•  Fallacy  3:  Skill  learning  is  intrinsically  enjoyable 

•  Fallacy  4:  Train  for  accurate  performance 

•  Fallacy  5:  Initial  performance  is  a  good  predictor  of  trainee  and  training  program  success 

•  Fallacy  6:  Once  learner  has  a  conceptual  understanding  of  the  system,  proficiency,  will 
develop  in  the  operational  setting. 

9.  Training  studies  show  support  for  such  fallacies,  as  summarized  below: 
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a.  Practice  Makes  Perfect 

•  Some  studies  show  that  practice  on  operational  tasks  does  not  necessarily  improve 
performance.  Sometimes  no  improvement  is  observed.  It  has  been  demonstrated  that 
practice  on  consistent  elements  of  a  task  does  improve  performance  (i.e.  consistent 
relationship  between  stimulus  and  response  elements). 

b.  Training  of  the  Total  Skill 

•  Training  in  “real  situations”  or  those  with  very  high  fidelity  do  not  necessarily  lead  to  the 
best  training  and  performance.  During  performance  of  a  real-world  task,  the  learner 
(particularly  the  novice)  is  often  overloaded  and  may  not  encode  and  retain  needed 
information  for  learning  and  performance  improvement.  Often,  a  learner  may  improve 
learning  by  practicing  parts  of  the  total  task  and  then  practicing  the  whole  task  at  a  later 
training  progression. 

•  Practice  on  consistent  component  tasks  does  improve  component  skills  (p.  287). 

•  In  many  situations  there  is  beneficial  results  (positive  transfer)  achieved  with  part-task 
training  of  key  skill  components. 

c.  Skill  Learning  is  Intrinsically  Enjoyable 

•  Failure  rates  in  training  can  be  improved  with  better  incentives  and  improved 
performance  feedback. 

d.  Train  for  Accurate  Performance 

•  In  most  cases  it  is  better  to  achieve  acceptable  accuracy  while  having  the  learner  pay 
attention  to  critical  tasks  and  important  environmental  cues. 

•  Over  training  may  be  desirable  in  cases  that  require  a  performer  to  work  in  a  high 
workload  operational  environment. 

e.  Initial  performance  is  a  Good  Predicator  of  Trainee  and  Training  Effort  Success 

•  Most  initial  performance  is  highly  unstable  and  not  a  good  indicator  of  ultimate 
performance  of  complex  skills.  The  correlation  between  early  performance  scores  and 
later  performance  is  often  very  low. 

•  Many  studies  show  that  augmented  feedback  may  facilitate  performance  during  training 
but  may  actually  slow  learning. 

f.  Once  Learner  has  Conceptual  Understanding  of  the  System.  Proficiency  will  Develop  in 

the  Operational  System 

•  Technical  programs  based  substantially  on  only  classroom  teaching  typically  fail. 

•  There  is  no  substitute  for  hands-on  experience  with  the  operational  system.  Learning  a 
complex  skill  continues  throughout  an  operator’s  experience. 

APPLICATION  OF  FINDINGS 

•  Trainers  should  focus  on  arranging  practice  on  consistent  task  components. 

•  Complex  tasks  can  be  broken  down  into  simpler  components  for  part-task  training  with 

positive  results. 
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•  Trainers  should  give  considerable  thought  to  sequencing  of  the  tasks  to  be  trained,  and  the 
kinds  of  performance  feedback  that  may  or  may  not  be  appropriate  at  various  learning 
phases. 

RELATED  REFERENCES: 

Fisk,  A.D.,  &  Gallini,  J.K.  (1989).  Training  consistent  components  of  tasks:  Developing  an 
instructional  system  based  on  automatic/controlled  processing  principles.  Human  Factors.  3 1 , 
453-63. 

Myers,  G.L.,  &  Fisk,  A.D.,  (1987).  Training  consistent  task  components:  Application  of 
automatic  and  controlled  processing  theory  to  industrial  task  training.  Human  Factors,  29,  (3), 
255-268. 

Schneider,  W.,  &  Shiffrin,  R.M.  (1977).  Controlled  and  automatic  human  information 
processing:  Detection,  search,  and  attention.  Psychological  Review.  84,  1-66. 
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REFERENCE  SYNOPSIS 

Smith,  B.A.  (1984).  The  optimum  presentation  of  cognitive  training  during  flight  training 
program.  In,  Proceedings  of  the  Human  Factors  Society — 28th  Annual  Meeting. 

ABSTRACT:  Experiments  were  conducted  while  training  flight  maneuvers  on  a  General 
Aviation  Trainer  for  beginning  aviation  students.  Instruction  on  cognitive  components  of  the 
flight  task  was  given  at  different  times  during  the  course  of  training.  The  study  used  thee 
experimental  treatment  groups  and  one  control  group  to  study  the  effects  of  cognitive 
presentation  methods.  One  treatment  group  received  cognitive  instruction  in  the  traditional 
manner,  given  just  prior  to  receiving  any  hands-on  flight  training.  A  second  treatment  group 
received  the  same  cognitive  instruction  prior  to  commencing  simulation  trials  but  then  got 
additional  cognitive  instruction  throughout  the  entire  training  period.  A  third  group  did  not 
receive  any  cognitive  training  before  participating  in  simulation  training,  but  did  get  cognitive 
instruction  during  simulation  trials,  and  finally  a  fourth  group  served  as  a  control  in  which  no 
cognitive  instruction  was  presented  before  or  during  simulation  trials.  The  results  of  the  study 
showed  that  subjects  that  only  received  cognitive  instruction  prior  to  commencing  simulation 
trials  performed  better  during  acquisition  and  later  transfer  trials.  Those  that  received  cognitive 
instruction  only  during  simulation  trials  performed  significantly  below  the  other  experimental 
groups,  and  below  the  control  group.  Results  indicated  that  the  presentation  of  cognitive 
instruction  during  simulation  practice  might  in  fact  impair  learning  and  transfer  of  performance 
on  the  maneuver  studied. 

KEY  TOPICS  AND  ISSUES: 

The  sequencing  of  instruction  involving  the  presentation  of  either  cognitive  or  perceptual 
components  of  a  high-performance  skill  may  have  an  important  bearing  on  learning 
effectiveness. 

•  Cognitive  components,  regarding  the  structure  of  the  task,  instructions  about  correct 
task  order,  equipment  capabilities  and  use,  and  instrument  readings,  etc.  are  best 
taught  prior  to  commencing  simulation  trials. 

•  The  best  retention  of  perceptual  components,  regarding  search  patterns,  cue 
distinctions,  visual  and  kinesthetic  feedback  occurs  when  subjects  are  directly 
exposed  to  the  flight  environment  and  allowed  to  develop  their  own  strategies  during 
practice,  without  interference  from  external  cognitive  instruction. 

APPLICATION  OF  FINDINGS 

•  When  arranging  simulation  instruction,  trainers  should  consider  the  optimum  presentation  of 
cognitive  instruction  to  facilitate  learning  and  transfer  of  training. 

•  It  may  be  most  beneficial  to  offer  cognitive  approaches  early  in  training,  and  then  shift  to 
perceptual-motor  aiding  later  in  the  training  period. 
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•  Providing  cognitive  instruction,  and  “over-coaching”  during  hands-on  equipment  or 

simulation  training  may  actually  interfere  with  skilled-performance  and  with  learning  some 
complex  skills. 

RELATED  REFERENCES: 

Adams,  J.A.  (1969).  Motor  behavior.  In,  M.H.  Marx  (Ed.),  Learning  Processes,  pp.  481-507. 
New  York:  McMillan. 

Fisk,  A.D.  &  Schneider,  W.  (1981).  Control  and  automatic  processing  during  tasks  requiring 
sustained  attention:  A  new  approach  to  vigilance.  Human  Factors.  23,  737-750. 

Fleishman,  E.A.  (1972).  On  the  relation  between  abilities,  learning  and  performance.  American 
Psychologist,  27,  1017-1032. 
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REFERENCE  SYNOPSIS 

Wightman,  D.C.,  &  Sistrunk,  F.  (1987).  Part-task  training  strategies  in  simulated  carrier  landing 
final-approach  training.  Human  Factors.  29.  (29),  245-254. 

ABSTRACT:  Two  part-task-training  strategies  were  tested  during  a  carrier  landing  final 
approach  task.  The  first  strategy  used  segmentation  (chaining)  and  the  second  strategy  used  task 
simplification  (enhancement  of  the  simulated  aircraft’s  response  to  throttle  adjustment). 
Performance  on  a  video  game  was  used  to  test  motor-skills  aptitude.  The  backward  chaining 
approach  (in  which  the  last  or  terminal  task  is  taught  first)  produced  the  best  transfer  to  the 
criterion  task  compared  to  training  on  the  criterion  task  itself.  Aptitude  treatment  interactions 
were  significant,  indicating  that  the  low- aptitude  subjects  benefited  most  from  the  backward 
chaining  part-task-training  method. 

KEY  TOPICS  AND  ISSUES 

1 .  Carrier  landing  is  a  task  very  suitable  to  part-task  training  strategies  using  both  segmentation 
and  simplification  techniques. 

•  Segmentation  is  a  procedure  that  partitions  the  criterion  task  into  spatial  or  temporal 
parts.  An  example  is  backward  chaining  in  which  the  final  segment  of  a  task  like  landing 
is  practiced  first  and  earlier  task  segments  are  progressively  added  during  later  training 
trials. 

•  Simplification  is  a  procedure  in  which  a  complex  task  is  made  easier  by  adjusting  or 
controlling  specific  characteristics  to  simplify  performance.  For  example,  reducing 
control-display  lag  in  a  tracking  system  (Wightman  and  Lintem,  1985) 

2.  Results  of  this  experiment  corroborated  earlier  finding  from  Bailey,  Hughes,  and  Jones 
(1980)  which  reported  success  using  a  backward  chaining  approach  for  training  air-to-ground 
bomb  attacks. 

•  Subjects  trained  under  the  chaining  approach  outperformed  those  trained  on  the  whole 
task  during  transfer  tests.  Fewer  errors  were  made  by  the  segmented  training  group  so 
this  treatment  resulted  in  more  training  trials  in  which  subjects  were  able  to  accurately 
perform  their  landing  task. 

•  Performance  observed  during  acquisition  and  transfer  also  indicated  that  low-aptitude 
subjects  benefited  most  from  the  segmented  part-task-training  strategy. 

•  The  use  of  controlled  enhancements  to  aircraft  responsiveness,  as  a  part-task-training 
strategy  was  not  found  to  be  effective. 

APPLICATION  OF  FINDINGS 

1 .  Results  reported  showed  that  learning  was  more  effective  if  subjects  make  fewer  errors 
during  training  trials.  Learners  using  the  segmented  approach  made  fewer  recorded  errors. 
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2.  The  backward  chaining  approach  also  helps  the  learner  focus  on  the  correct  performance  of 
the  terminal  task  (in  this  case  the  final  landing  task)  without  the  “ambiguities  resulting  from 
the  accumulation  of  errors  on  previous  task  segments. 

RELATED  REFERENCES 
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ATTACHMENT  B 

PEDAGOGICAL  KNOWLEDGE  BASE  SUMMARY 

General  Prescriptions: 

•  Provide  students  with  tutorial  information  regarding  the  task  structure  and  salient  cues  to  pay 
attention  to  during  task  performance. 

•  Demonstrate  correct  performance  to  student,  showing  correct  response  sequence  and  timing. 

•  Reinforce  attention  to  salient  cues  during  practice  sessions,  and  provide  diagnostic  feedback 
of  performance  results. 

•  As  skill  becomes  more  automated,  provide  less  coaching  and  verbal  support  during  task 
performance. 

•  It  may  be  most  beneficial  to  offer  cognitive  approaches  to  skill  learning  early  in  training,  and 
then  shift  to  perceptual-motor  aiding  later  in  the  training  period. 

•  Learner  should  be  given  a  relevant  organizational  introduction  to  the  task  domain  prior  to 
engaging  in  practice  trials,  and  then  more  emphasis  should  be  placed  on  task  specific 
information  and  couching. 

•  Providing  cognitive  information  and/or  verbal  feedback  during  practice,  for  trainees  who 
have  reached  the  level  of  skill  automaticity,  may  actually  interfere  with  learning. 

Specific  Prescriptions  based  on  Literature  Review 

1 .  Allow  learner  to  practice  consistent  task  components  over  many  training  trials  (Schneider, 
1985). 

2.  Control  task  difficulty  and  information  flow  as  to  not  overload  short-term  memory  during 
early  learning  acquisition  trials  (Schneider,  1985). 

3.  Add  variation  to  practice  progressively,  during  later  training  phases,  to  represent  anticipated 
inconsistencies  in  the  operational  environment  (Schneider,  1985). 

4.  Present  cognitive  aspects  of  complex  skills  early  in  the  training  cycle,  before  substantial  skill 
automation  (Fitts,  1967;  Smith,  1981). 

5.  Adjust  task  difficulty  to  minimize  learner  errors  and  to  ensure  exposure  to  perceptual  cues 
associated  with  correct  task  performance  (Wightman  and  Sistrunk,  1987). 

6.  Allow  learner  some  time  on  simulation  task  that  permits  “free  play”  experimentation  in  order 
to  permit  the  learner  to  develop  individual  strategies  (Smith,  1981). 

7.  Complex  tasks  can  be  broken  down  into  simpler  components  for  part-task  training  with 
positive  results  (Schneider,  1985). 

8.  Arrange  order  of  instruction  such  that  knowledge  and  skill  prerequisites  are  met  before 
progressing  to  more  difficult  training  sessions  (Caro,  1973). 

9.  Feedback  on  good  performance  is  as  important  as  feedback  on  poor  performance  (Schimmel, 
1988). 

10.  Clearly  relate  the  learning  objectives  (task,  conditions,  and  performance)  to  the  operational 
environment  (task  demands,  physical  environment,  information  etc.). 
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1 1 .  Allow  for  self-paced  practice  and  provide  knowledge  of  results  feedback,  especially  during 
early  training  trials  (Swezey  &  Llaneras,  1997). 

12.  Instructional  designers  should  identify  context  (environmental)  cues  that  are  the  most 
powerful  in  capturing  attention  and  initiating  correct  or  incorrect  responses  during  later 
periods  of  training  (Wightman  and  Sistrunk,  1987). 

13.  Learning  is  more  effective  if  subjects  make  fewer  errors  during  training  trials. 

14.  Learners  using  the  segmented  approach  made  fewer  recorded  errors  (Wightman  and  Sistrunk, 
1987)  compared  to  other  part-task  strategies. 

15.  The  backward  chaining  part-task  training  strategy  works  because  it  helps  the  learner  focus  on 
the  correct  performance  of  the  terminal  task  without  the  “ambiguities  resulting  from  the 
accumulation  of  errors  on  previous  task  segments”  (Wightman  and  Sistrunk,  1987). 
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Purpose 

The  US  Army  is  interested  in  reducing  the  cost  of  training  students  to  hover  a  helicopter  In  pursuit  of  this  goal,  the  Office  of  the  Secretary  of 
Defense  (OSD)  is  sponsoring  the  Small  Business  Innovative  Research  (SBIR)  contract  OSD99-04  to  enhance  the  Army  Research  Institutes’ 

(ARI)  Intelligent  Flight  Trainer  (IFT).  The  desired  enhancement  would  automatically  adapt  the  IFT  to  facilitate  student  learning  in  the  absence  of 
an  instructor. 

This  report  summarizes  an  effort  to  identify  and  determine  the  feasibility  of  a  technical  approach  to  implementing  the  IFT  training  enhancement. 
Following  a  brief  description  of  the  IFT  and  the  general  solution  architecture,  several  commercial  software  products  are  reviewed  as  candidates 
for  inclusion  in  the  final  system  implementation.  The  report  concludes  with  the  recommended  system  architecture,  knowledge  acquisition 
approach,  and  required  software  components. 

The  Intelligent  Flight  Trainer  (IFT) 

The  IFT  is  currently  described  as  an  automated  and  adaptive  helicopter  instructor  pilot  providing  verbal  instruction  to  novice  pilots.  It  also  has 
the  ability  to  augment  control  inputs  until  the  student  is  able  to  accomplish  a  given  maneuver  unaided.  While  this  description  seems  to  satisfy  the 
requirements  for  an  automated  hover  trainer,  it  does  not  adequately  adapt  the  instruction  provided  to  the  experience  level  of  the  student  pilot. 

The  [FT  consists  of  several  cooperating  PC-based  computer  systems  The  main  system  is  a  Dell  610  Dual  Processor  machine  running  LINUX 
that  serves  as  the  central  controller.  It  hosts  the  software  responsible  for  the  helicopter  flight  model,  instrument  panel  graphics,  and  adaptive 
instructor  functions.  Figure  1  presents  a  schematic  of  the  IFT 


Figure  1:  General  Schematic  of  the  IFT 
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A  second  Dual  Processor  system  running  Windows  NT  is  used  to  generate  the  two  “out-the-window”  views  displayed  to  the  student.  The 
graphics  are  generated  on  accelerated  displays  using  the  OpenGVS  simulation  toolkit.  The  IFT  also  maintains  the  ability  to  drive  an  ES1G  3000 
Image  Generator.  A  third  machine  running  MS-DOS  is  used  to  generate  synthesized  speech  instructions. 

In  addition  to  the  two  “out-the-window”  views,  the  main  system  generates  two  other  screens  of  information  The  first  screen  generated  is  an 
operator  station  display  that  provides  menu  control  over  the  system  and  access  to  flight  parameters.  The  second  screen  is  positioned  in  front  of 
the  pilot  and  presents  flight  instrumentation  using  the  TIGERS  software  package 

Adaptive  Training  System  (ATS)  Requirements 

The  initial  proposal  for  SBIR  OSD99-04  proposed  to  build  an  Adaptive  Training  System  (ATS)  that  is  comprised  of  several  components.  The 
following  identifies  the  high  level  components  that  make  up  the  general  ATS 

1 .  Model  of  the  Student  Pilot  (SP), 

2.  Model  of  the  Instructor  Pilot  (IP), 

3.  Knowledge  Acquisition  Systems  (KAS),  and 

4.  Interfaces  between  the  ATS  and  the  IFT. 

The  SP  model  encapsulates  all  knowledge  and  processing  required  in  estimating  the  student's  skill  state  and  task  understanding.  The  IP  model 
encapsulates  the  reasoning  systems  required  to  diagnose  student  deficiencies  and  recommend  training  remedies.  The  Knowledge  Acquisition 
Systems  (KAS)  facilitate  the  construction  and  maintenance  of  the  SP  and  IP  models  Finally,  appropriate  connections  between  the  ATS  system 
and  the  IFT  must  be  established. 

Student  Pilot  (SP)  Model  Requirements 

The  SP  model  as  described  here  provides  a  standalone  evaluation  of  a  student  helicopter  pilot  in  the  IFT  The  evaluation  should  employ  current 
performance  data  and  the  student's  history  with  the  IFT.  Two  primary  areas  of  evaluation  should  be  modeled:  task  understanding  and  skill  state. 
Skill  state  describes  a  level  of  perceptual-motor  performance  against  a  given  control  task  in  the  IFT  under  specific  conditions.  To  evaluate  a 
student's  skill  state,  the  SP  model  must  represent  goal  performance  for  the  given  task  and  determine  the  skill  ranking  using  measures  obtained 
from  the  IFT.  The  following  is  an  example  of  a  task  and  potential  skill  state  metrics 


Task 

Maintain  Lateral  Position 

Criteria 

+/-  5  feet  from  designated  spot 

Conditions 

■5  knot  crosswind,  gusting  to  15  knots 

Metrics 

"Average  error",  "maximum  error",  "number  of  deviations”,  "frequency  of 
deviations",  and  “duration  of  deviations" 

Table  1:  Sample  Student  Task 

Quantifying  a  student's  "understanding”  of  a  task  is  more  difficult  than  determining  skill  state  In  simple  cases,  inferior  performance  on  a  specific 
task  may  clearly  indicate  a  lack  of  "understanding"  with  regard  to  what  is  required  to  accomplish  the  task.  However,  composite  tasks  that  have 
several  objectives  may  lead  to  contrary  indications  of  "understanding" 

Instructor  Pilot  (IP)  Model  Requirements 

An  instructor  pilot  begins  with  a  student  and  a  training  goal.  The  instructor  assesses  the  student's  skill  level  with  regard  to  the  training  goal  and 
develops  a  training  plan.  Armed  with  a  training  plan,  the  instructor  engages  the  student  in  a  series  of  activities  intended  to  provide  an  efficient 
context  for  acquiring  the  required  skill  level. 
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The  training  engagements  provide  additional  opportunities  for  assessing  the  student’s  skill  level  and  identifying  any  difficulties  in  accomplishing 
specific  tasks.  As  difficulties  are  encountered,  the  instructor  must  attempt  to  diagnose  the  cause  of  the  student’s  trouble.  Once  the  problem  is 
diagnosed,  the  training  plan  can  be  altered  to  address  the  newly  discovered  information  needs  of  the  student. 

Timely  adaptation  of  the  training  plan  to  the  needs  of  the  student  is  critical  in  providing  efficient  instruction.  Frustrations  over  not  being  able  to 
perform  a  task  can  be  as  irritating  as  working  on  skills  that  have  already  been  mastered.  Frustration  might  indicate  a  requirement  for  remedial 
training  and  suggest  a  postponement  of  the  currently  planned  "practice".  On  the  other  hand,  perfect  performance  may  suggest  an  acceleration  of 
the  training  plan  schedule. 

Knowledge  Acquisition  Systems  (KAS)  Requirements 

The  first  rule  in  building  an  Expert  System  is  that  there  must  be  at  least  one  expert  available  during  the  construction  of  the  system.  In  practice, 
this  is  perhaps  the  most  frequently  broken  rule  in  Expert  System  development.  The  Subject  Matter  Expert  (SME)  in  a  field  is  often  in  high 
demand  and  not  always  available  when  needed. 

To  stretch  the  SME  budget  a  little  farther,  several  experts  are  generally  consulted  over  the  period  of  a  systems  development  to  capture 
specializations.  Many  experts  contributing  knowledge  to  a  system  over  a  longer  time  drives  the  need  for  an  interchangeable  knowledge  structure. 
The  successful  approach  to  developing^and  maintaining  the  Adaptive  Training  System  (ATS)  should  include  one  or  more  Knowledge  Acquisition 
Systems  to  help  maintain  and  formalize  acquired  knowledge. 

The  KAS  should  support  developers  and  experts  in  the  accomplishing  the  following  tasks  related  to  knowledge-based  system  development: 

•  Select  appropriate  representations  for  knowledge, 

•  Organize  knowledge  and  manage  changes  to  the  knowledge  base, 

•  Explain  the  purpose  for  specific  knowledge  structures, 

•  Identify  relationships  between  cooperating  knowledge  structures. 

The  tools  selected  to  implement  knowledge  structures  should  address  the  following  implementation  and  maintenance  concerns: 

•  Representation  portability  between  components, 

•  Heterogeneous  interoperability, 

•  Automated  regression  testing, 

•  Scalable  repository  management. 


Required  Interfaces  between  the  ATS  and  the  IFT 

In  order  for  the  ATS  to  drive  the  IFT,  control  aspects  within  the  IFT  must  be  exposed  to  ATS.  In  addition  some  data  items  maintained  by  the  IFT 
must  be  made  available  to  the  ATS.  With  the  exception  of  environmental  perturbations,  all  of  the  required  data  from  the  IFT  is  currently  output 
to  a  file  every  60  frames.  This  output  mechanism  can  be  altered  to  provide  data  to  the  ATS.  The  following  table  identifies  inputs  needed  from 


ATS  Metric 

Required  Input  from  IFT 

Control  Position  &  Displacement 
Frequency 

Time  stamped  axis  value  and  flight  model  calibration  tables. 

Vehicle  Degrees  of  Freedom 

Time  stamped  position,  velocity,  acceleration,  orientation,  angular 
velocity,  and  angular  acceleration  vectors  for  the  vehicle 

Flight  Handling  Qualities 

Augmentation 

Alterations  to  the  base  flight  model.  Sensitivity  damping,  axis 
reduction  and  any  other  FHQ  Augmentation. 

Environmental  Perturbations 

Time  stamped  wind  velocity  and  direction.  Visibility  attenuation  and 
scale. 

Table  2:  ATS  Metrics  and  Required  IFT  Inputs 
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In  order  to  reuse  all  existing  simulation  capabilities  encapsulated  within  the  1FT,  control  aspects  that  are  currently  within  the  IFT  must  be  exposed 
for  use  by  ATS.  The  following  list  identifies  control  aspects  of  the  IFT  that  are  needed  by  the  ATS  : 

1.  Flow  control:  The  remote  initiation  and  termination  of  the  IFT 

2.  Remote  flight  control:  The  ability  to  bypass  the  IFT  flight  control  augmentation  and  pass  ATS  flight 
control  values  to  the  flight  model 

3.  Remote  voice  control:  The  ability  to  bypass  the  IFT  verbal  instructions  and  pass  the  ATS  verbal 
instructions  to  the  voice  synthesizer. 

4.  Remote  environmental  control:  The  ability  to  pass  ATS  commanded  wind  and  visibility  control  to  the  IFT 
flight  mode!  and  graphics  systems. 


Candidate  Technologies 

The  technologies  identified  in  the  section  were  evaluated  for  use  in  implementing  the  ATS  In  general,  these  technologies  identify  knowledge 
representations  for  solving  specific  types  of  problems.  Where  appropriate,  ATS  requirements  are  mapped  to  the  considered  technology  to  direct 
candidate  product  reviews. 

Conceptual  Graph  Analysis 

Conceptual  graphs  (CGs)  are  a  system  of  logic  based  on  the  existential  graphs  of  Charles  Sanders  Peirce  and  the  semantic  networks  of  artificial 
intelligence.  Meaning  is  expressed  in  a  logically  precise,  humanly  readable,  and  computationally  tractable  form.  Direct  mappings  to  language 
and  the  common  Knowledge  Interchange  Formats  (KIF)  make  CG's  useful  in  developing  and  maintaining  knowledge-based  systems. 

Conceptual  Graph  Standard 

Figure  2  presents  a  graphical  representation  of  the  concept  "Tom  believes  that  Mary  wants  to  marry  a  sailor".  This  concept  demonstrates  context 
by  depicting  the  nested  concepts  of  Proposition  and  Situation.  It  also  demonstrates  a  co  reference  link  between  Mary  and  her  surrogate 
instantiation  in  the  nested  concept  Situation. 
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Figure  2:  Conceptual  Graph  Example 


The  Conceptual  Graph  Interchange  Format  (CGIF)  for  Figure  4  is: 

1 Person :  *xl  'Tom'/  [Believe  *x2[  (Expr  ?x2  ?xl) 

(Thme  ?x2  [Proposition: 

[Person:  *x3  'Mary'/ [Want  *x4[(Expr  ?x4  2x2) 
jj  (Thme  ?x4  [ Situation : 

f  [Marry  *x5J  (Agnt  2x5  2x2)  (Thme  2x5 /Sailor/)  [)  [), 

and  the  Knowledge  Interchange  Format  (KIF)  is  : 

(exists  ((?xl  person)  (?x 2  believe)) 

(and  (expr  ?x2  ?xl) 

S  (thme  ?x2 

(exists  ((2x2  person)  (?x4  want)  (?x8  situation)) 

(and  (name  ?x3  ’Mary)  (expr  ?x4  ?x3)  (thme  ?x4  2x8) 

(dscr  ?x8  (exists  ((?xS  marry)  (?x6  sailor)) 

(and  (Agnt  ?x5  2x2)  (Thme  2x5  2x6))))))))) 

Conceptual  Graphs  and  ATS 

The  ability  to  create  and  manage  conceptual  relations  in  a  variety  of  formats  provides  a  robust  basis  for  knowledge  acquisition  and  knowledge 
structure  explanation.  Tool  selection  should  provide  for  a  convenient  flow  of  representations  between  human  and  machine-readable  forms.  The 
Conceptual  Graph  Structure  provides  a  standard  representation  for  connecting  systems  with  a  variety  of  internal  knowledge  representation 
structures. 

Assuming  that  ATS  will  use  several  products  to  develop  and  maintain  the  system,  CG's  could  serve  as  a  representation  for  knowledge  acquisition 
products  generated  during  interviews  with  helicopter  instructors.  The  CG  tool  should  support  the  real-time  interrogation  of  existing  knowledge 
structures  in  addition  to  generating  new  structures.  As  instructors  offer  expertise,  the  knowledge  engineer  needs  the  capability  to  identify 
redundant,  conflicting,  or  new  concepts.  If  conflicting  concepts  are  discovered,  this  current  knowledge  acquisition  session  may  represent  the  only 
opportunity  to  unravel  the  source  of  the  conflict. 

Over  time,  instructors  encounter  students  with  a  wide  variety  of  training  difficulties  Expert  instructors  anticipate  these  difficulties  based  on  their 
experience  with  other  students  of  similar  skill  level.  The  training  of  an  instructor  involves  building  up  a  network  of  discrimination  concepts  that 
relate  the  student’s  skill  level  to  the  types  of  performance  problems  that  the  student  may  have  with  a  given  task. 


Case  Based  Reasoning  (CBR) 

Case-Based  Reasoning  (CBR)  is  a  software  technology  that  supports  decision-making  and  problem  resolution  by  leveraging  prior  experience.  In 
CBR,  a  particular  experience  is  represented  in  a  database  by  one  or  more  cases.  Any  number  of  attributes  describing  the  specific  information 
detailing  a  case  is  remembered  and  used  to  identify  this  case  as  relevant  in  solving  some  future  problem. 
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The  Reasoning  part  of  CBR  involves  the  representation  of  an  "understanding”  of  the  problem  being  solved.  This  is  required  to  allow  the  CBR 
system  to  adapt  prior  experiences  to  the  current  problem  context.  In  order  to  understand  a  prior  experience,  it  is  remembered  with  associative  or 
conceptual  relations.  These  relations  provide  a  means  for  the  Reasoning  part  of  the  CBR  system  to  retrieve  relevant  cases. 

Once  relevant  cases  are  retrieved,  an  understanding  of  the  differences  between  the  prior  experience  and  the  current  problem  are  used  to  select  the 
best-fit  cases.  A  solution  that  worked  in  the  past  is  then  adapted  to  generate  a  new  solution  for  the  current  case.  If  the  retrieved  cases  are 
irrelevant,  a  new  case  is  added  once  the  new  problem  has  been  solved  by  traditional  means. 

Benefits  from  Using  Case-Based  Technologies  include: 

•  Finding  solutions  to  complex  problems  more  quickly, 

•  Discovering  decision  knowledge  hidden  in  data  using  induction, 

•  Transferring  experience  from  skilled  specialists  to  novices, 

•  Building  a  .corporate  memory  by  sharing  individual  experience,  and 

•  Demonstrating  expertise  in  domains  that  are  poorly  understood  because  CBR  does  not  need  to  know  why 
a  solution  worked  in  the  past. 

The  automatic  generation  of  decision  trees  from  a  database  of  stored  cases  is  called  induction.  Given  a  set  of  cases  with  problem  specific 
relevance  measures  and  existing  solutions,  a  decision  tree  can  be  developed  that  will  classify  the  existing  case-base  along  some  decision  concept. 
The  generated  decision  tree  can  then  be  used  to  classify  and  suggest  solutions  for  completely  new  problems  without  consulting  the  case-base. 

It  is  important  to  realize,  however,  that  induced  decision  trees  treat  all  new  cases  as  if  they  "match"  one  of  the  prior  cases.  Since  the  decision  tree 
was  synthesized  from  the  case-base,  decision  trees  must  be  re-generated  when  important  changes  to  the  case-base  are  stored. 

In  addition  to  using  the  generated  decision  tree  to  "speed  up"  ATS  response  to  new  cases,  the  induced  decision  tree  could  be  employed  during 
training  concept  discovery.  The  decision  tree  produced  from  a  case-based  rule  induction  results  in  a  generalized  representation  of  the  underlying 
specific  instances.  This  automated  generalization  process  could  result  in  an  efficient  validation  and  verification  mechanism  for  ATS  knowledge 
engineers. 

Modeling  and  Simulation 

Modeling  and  simulation  (M&S)  is  a  general  category  of  technology  that  attempts  to  utilize  known  relationships  to  predict  the  behavior  of  a 
system.  Designers  employ  simulations  of  proposed  systems  to  estimate  overall  differences  between  design  alternatives.  Human  Factors 
researchers  use  M&S  to  estimate  the  affects  of  design  alternatives  on  human  operators.  Finally,  the  simulations  can  be  used  to  train  the  human 
operators,  as  is  the  case  with  the  IFT. 

While  the  ATS  is  specifically  designed  to  control  a  helicopter  hover  training  simulator,  it  could  itself  be  viewed  as  a  training  simulator  for 
helicopter  instructors.  An  instructor  could  observe  the  behavior  of  the  system  and  request  explanations  regarding  its  decision-making 

The  type  of  M&S  considered  here  supports  knowledge  engineering  activities  focused  on  the  development  of  the  ATS.  The  M&S  function  that 
would  be  most  useful  in  knowledge  base  construction  and  concept  discovery  is  a  model  of  the  student  helicopter  pilot.  A  collection  of  models 
ranging  from  novice  to  expert  would  provide  test  cases  to  exercise  all  aspects  of  the  completed  ATS. 

Neural  Networks  and  Genetic  Algorithms 

Neural  Networks  are  collections  of  nodes  that  are  analogous  to  neurons  in  the  brain  The  nodes  are  interconnected  in  a  network  that  has  the 
potential  to  identify  patterns  in  data.  Each  of  the  nodes  are  processing  elements  that  represent  weighting  factors  for  each  of  the  interconnections. 
When  training  sets  are  combined  with  a  learning  strategy  for  adjusting  the  weighting  factors,  networks  can  be  developed  that  can  learn  almost  any 
function  regardless  of  noise  in  the  data  or  function  complexity 

Neural  Network  (NN)  node  densities  and  training  sets  are  the  keys  to  developing  an  accurate  solution.  High  densities  for  simple  problems  result 
in  networks  that  learn  about  the  noise  in  the  training  sets.  Low  densities  for  complex  problems  result  in  networks  that  never  quite  get  "smart" 
enough.  In  addition,  at  least  three  training  sets  must  be  available:  the  training  set,  the  test  set,  and  the  validation  set.  The  test  set  is  used  during 
training  to  monitor  learning  performance  and  the  validation  set  is  used  to  determine  the  final  model  performance 

For  a  NN  to  perform  the  ATS  hover  instruction  task,  the  network  would  need  to  be  extremely  dense  due  to  the  large  number  of  input  and  output 
nodes.  Input  nodes  "sense"  information  from  the  environment  and  output  nodes  "react"  with  appropriate  solutions.  This  dense  network  would 
need  a  large  number  of  training  data  sets  to  "exercise"  all  portions  of  the  network 
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Instead  of  performing  the  entire  ATS  instruction  task  with  a  single  network,  perhaps  individual  networks  could  be  devised  that  worked  on  single 
aspects  of  the  instruction  task.  For  example,  a  single  network  might  classify  a  student's  skill  state  as  acceptable  or  not.  Others  might  classify 
certain  vehicle  motions  as  "out  of  bounds"  and  issue  a  warning. 

The  main  difficulty  with  Neural  Networks  (NN's)  is  in  trying  to  explain  their  behavior.  This  in  turn  makes  the  network  hard  to  maintain  All  you 
can  really  do  is  add  your  new  data  to  the  training  set(s)  and  retrain  the  network  This  incremental  increase  in  training  set  complexity  may 
eventually  overwhelm  the  existing  architecture,  forcing  an  adjustment  to  the  node  densities 

Genetic  Algorithms  (GA's)  are  algorithms  that  by  there  nature  evolve  into  solution  providers.  Populations  of  "infant"  algorithms  are  released  into 
a  problem  environment  and  rated  for  fitness.  New  populations  are  constructed  by  crossing  over  the  parents  to  create  new  offspring  and  then 
mutating  them  using  some  probabilities.  These  new  populations  are  released  back  into  the  problem  environment  and  the  process  continues  until 
an  end  condition  is  satisfied. 

If  you  were  thinking  that  it  might  take  a  while  to  train  a  neural  network  in  a  complex  environment,  imagine  the  time  required  to  evolve  a  solution 
to  providing  helicopter  instruction  to  a  human  subject.  As  before,  a  simplification  of  the  problem  environment  may  reduce  the  complexity,  but 
selecting  appropriate  pieces  of  the  problem  would  be  at  best  difficult  Other  issues  include  selecting  the  genetic  encoding  scheme,  specifying  the 
crossover  function,  and  the  mutation  function. 

For  ATS,  both  NN's  and  GA's  require  significantly  larger  supplies  of  test  data  than  other  approaches  to  implementing  expertise.  Neither  is  able  to 
supply  a  human  understandable  explanation  of  how  it  arrives  at  a  decision.  And  finally,  there  is  significant  risk  as  to  whether  the  problem  is 
solvable  given  the  selected  node  densities,  training  sets,  fitness  measures,  crossover  functions  and  mutation  approaches.  For  these  reasons,  NN 
and  GA  products  were  not  evaluated  as  candidates  for  employment  in  developing  the  ATS 


Candidate  Products 

CommonKADS  Methodology 

CommonKADS  does  not  claim  to  be  a  full  knowledge-management  methodology,  but  is  in  practice  used  successfully  as  a  powerful  tool  to 
support  knowledge  management.  The  analysis  framework  provides  an  extensive  method  for  describing  business  processes  in  which  knowledge- 
intensive  tasks  are  carried  out.  It  is  not  offered  specifically  as  a  tool,  but  is  instead  a  methodology  that  can  be  employed  using  standard 
organizational  toolsets. 

With  a  clear  focus  on  knowledge  analysis,  CommonKADS  provides  tool  methods  required  to  analyze  knowledge-intensive  tasks  at  different 
grain-size  levels.  The  analyst  is  supported  in  the  modeling  process  by  "templates”,  which  constitute  predefined  reusable  and  proven  knowledge 
models.  The  templates  enable  a  top-down  approach  and  provide  handles  for  quality  control  and  feasibility. 

Knowledge  analysis  is  aimed  at  studying  knowledge-intensive  tasks  at  a  conceptual  level.  The  analysis  results  in  a  description  of  the  information 
and  knowledge  structures  and  functions  involved  in  the  task  The  knowledge  model  plays  a  key  role  in  both  knowledge  management  work  and  in 
consecutive  system-development  activities.  Figure  3  presents  a  sample  template  for  the  "assessment"  task  used  to  decide  if  a  prospective 
homebuyer  should  be  eligible  to  buy  a  particular  residence. 

The  representation  techniques  employed  by  CommonKADS  are  similar  to  mainstream  object-oriented  design  paradigms.  In  fact,  the  diagram  in 
Figure  3  is  expressed  using  the  Universal  Modeling  Language  (UML),  a  standard  Object-Oriented  modeling  notation 
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Figure  3:  CommonKADS  "eligibility  assessment"  task 


While  this  type  of  detailed  modeling  is  desired  in  establishing  requirements  and  documenting  relationships  between  components,  it  does  not 
specifically  provide  management  software  tools.  CommonKADS  specifically  addresses  the  following: 

1 .  Analyzing  the  Existing  Infrastructure 

2.  Aligning  Knowledge  Management  and  Business  Strategy 

3.  Designing  the  Knowledge  Management  Infrastructure 

4.  Auditing  Existing  Knowledge  Assets  and  Systems 

5.  Designing  the  Knowledge  Management  Team 

6.  Creating  the  Knowledge  Management  Blueprint 

7.  Developing  the  Knowledge  Management  System 

8.  Deploying  and  Using  the  Results-driven  Incremental  Methodology 

9.  Managing  Change,  Culture  and  Reward  Structures 

1 0.  Evaluating  Performance,  Measuring  ROI,  and  Incremental  refinement. 

For  ATS,  the  main  problem  is  to  develop  a  system  that  adapts  its  own  behavior.  In  CommonKADS,  the  ATS  developer  can  model  existing 
knowledge  constructs,  but  is  not  directly  supported  in  the  production  of  the  implementation  vehicle.  For  projects  that  need  to  include 
considerations  about  knowledge-based  processing  into  a  standard  Object-Oriented  (OO)  Programming  cycle,  CommonKADS  would  be  useful.  It 
would  provide  a  good  OO  Analysis  and  Design  platform  from  which  to  launch  an  implementation  phase. 

For  ATS,  the  implementation  phase  represents  a  primary  source  for  knowledge  acquisition  activities.  Instructors  that  are  interviewed  for 
validation  of  training  sequences  will  be  queried  about  their  reactions  to  large  numbers  of  cases.  Synthesizing  their  responses  in  the  form  of 
conceptual  discriminators  will  be  a  challenge  to  knowledge  engineers.  We  need  a  suite  of  tools  that  provide  for  a  smooth  flow  of  conceptual 
discovery  from  the  implementation  environments  collection  of  experiences  and  not  just  a  good  OO  design  and  analysis  methodology.  Flowever, 
the  lifecycle  management  of  the  ATS  as  a  whole  must  consider  all  of  the  aspects  addressed  in  the  CommonKADS  methodology. 

KATE 

The  KATE  suite  of  software,  developed  by  AcknoSoft  International  (France),  is  a  collection  of  4  tools  for  developing  CBR  applications:  KATE- 
editor,  KATE-Data  Mining,  KATE-CBR  and  KATE-runtime.  Optional  components  are  available  for  specific  help  desk  application,  service 
management,  and  decision  support  over  the  Internet.  A  KATE  Dynamic  Link  Library  (DLL)  is  available  to  facilitate  integration  with  other 
applications. 
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KATE-Data  Mining  claims  to  be  the  fastest  rule  induction  algorithm  available  for  discovering  decision  knowledge  in  a  case-base.  KATE- 
induction  includes  a  graphical  browser  of  the  decision  tree  to  navigate  through  the  learned  knowledge  and  retrieve  cases  at 
any  node  It  also  provides  for  the  treatment  of  unknown  values  and  the  integration  of  background  knowledge. 


KATE-CBR  contains  two  modules:  the  nearest  neighbor  module  and  the  dynamic  induction  module.  The  nearest  neighbor  module  is  used  to 
compare  the  current  problem  with  ones  that  have  already  been  solved,  retrieve  the  most  similar  cases,  and  adapt  their  known  solutions.  The 
dynamic  induction  module  enables  discovery  of  the  most  discriminating  questions  and  retrieves  relevant  cases  efficiently.  Figure  4  presents  a 
sample  interface  for  the  CBR  tool. 


Figure  4:  KATE-CBR  Interface 

KATE-Editor  provides  for  the  object-based  modeling  of  cases  It  uses  the  object  model  to  generate  interactive  questionnaires  for  editing  the  case 
library.  The  editor  is  implemented  as  a  set  of  C  DLL's  to  allow  extension  to  user  interfaces.  KATE-runtime  provides  a  distributable  engine  for 
fielding  developed  solutions. 

The  CASSIOPEE  system,  awarded  the  1995  European  prize  for  innovative  software  applications,  was  developed  using  KATE.  The  system 
performs  diagnosis  of  the  CFM  56-3  engines  on  the  BOEING  737  It  contains  over  23.000  cases  of  which  70%  are  used  for  CBR  diagnosis. 

The  retail  price  of  KATE  is  $15,000  and  it  is  available  for  the  Windows  3  1. 95,  and  NT  3.51  and  4.0.  KATE  Webserver  and  the  KATE  libraries 
are  also  available  on  Sun  Solaris  and  soon  for  Linux  on  all  machines. 

KNOT 

The  Knowledge  Network  Organizing  Tool  (KNOT)  is  built  around  the  Pathfinder  network  (Pfnet)  generation  algorithm.  Pathfinder  algorithms 
take  estimates  of  the  proximities  between  pairs  of  items  as  input  and  define  a  network  representation  of  the  items.  The  network  consists  of  the 
items  as  nodes  and  a  set  of  links  connecting  pairs  of  the  nodes.  The  set  of  links  is  determined  by  patterns  of  proximities  in  the  data  and 
parameters  of  Pathfinder  algorithms. 

This  type  of  network  analysis  is  typically  employed  in  the  construction  of  neural  networks  Given  a  set  of  training  data,  users  manipulate  KNOT 
functions  to  analyze  relationships  between  nodes.  Concept  discovery,  however,  could  be  viewed  as  a  similar  search  for  the  nodes  in  a  graph 
given  some  known  relationships.  The  following  functions  are  provided  in  the  KNOT  system: 


•  Collect  pairwise  rating  data. 

•  Make  a  nonsymmetric  matrix  symmetric 

•  Average  multiple  data  files. 

•  Compute  a  coherence  measure  on  proximity  data 
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•  Correlate  pairs  of  proximity  data  sets. 

•  Generate  Pathfinder  networks  from  proximities 

•  Compute  distances  in  PFnets. 

•  Compute  the  similarity  of  two  PFnets 

•  Compute  node  positions  for  a  display  of  a  PFnet 

•  Handle  multiple  data  files. 

•  Display  a  PFnet. 

•  Move  nodes  to  new  positions  (links  follow) 

•  Print  a  PFnet. 

•  Create  and  erase  nodes  or  links 

•  Edit  node  labels  (multiple  line  labels) 

•  Display  directed  links. 

•  Handle  directed  links  in  a  display. 

The  KNOT  system  is  oriented  around  producing  pictures  of  the  solutions,  but  representations  of  networks  and  other  information  are  only 
available  in  the  form  of  text  files.  KNOT  may  be  useful  to  knowledge  engineers  as  an  offline  analysis  tool,  but  does  not  provide  any  directly 
useable  connections  to  other  technologies.  Once  ATS  has  represented  a  first-cut  at  conceptual  relationships  in  the  training  data,  KNOT  could 
assist  in  validating  those  relationships  or  perhaps  discover  "better"  relationships 

ServiceSoft  2001  (formerly  Knowledge  Builder) 

ServiceSoft  2001  is  a  Case-based  reasoning  (CBR)  tool.  It  provides  turnkey  CBR  solutions  to  service  related  problem  solving.  It  is  a  classical 
CBR  tool  focusing  exclusively  on  providing  support  for  the  retrieval,  reuse,  revision,  and  retention  of  cases.  Figure  5  presents  the  companies' 
product  in  the  context  of  integrated  platforms  and  applications. 
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Figure  5:  ServiceSoft  2001  Application  Architecture 


The  old  Knowledge  Builder  functionality  is  still  behind  the  scenes  helping  knowledge  engineers  implement  the  initial  CBR  implementation.  The 
Knowledge  Builder  product  is  no  longer  sold  independently,  but  is  incorporated  into  all  Servicesoft  2001  products.  For  ATS,  we  would  have  to 
model  the  behavior  as  a  service  application  along  the  lines  of  one  of  the  internet-based  ServiceSoft  products. 


SBIR  DATA  RIGHTS  Contract  No.  DASW01-00-M-4084.  Contractor  Name:  Monterey  Technologies,  Inc.,  Address:  26400  Silver  Cloud  Ct.,  #103, 
Monterey,  CA  93940.  Expiration  of  SBIR  Data  Rights  Period:  July  1 , 2005.  The  Government’s  rights  to  use,  modify,  reproduce,  release,  perform, 
display,  or  disclose  technical  data  or  computer  software  marked  with  this  legend  are  restricted  during  the  period  shown  as  provided  in  paragraph 
(b)(4)  of  the  Rights  in  Noncommercial  Technical  Data  and  Computer  Software- Small  Business  Innovation  Research  (SBIR)  Program  clause 
contained  in  the  above  identified  contract.  No  restrictions  apply  after  the  expiration  date  shown  above.  Any  reproduction  of  technical  data, 
computer  software,  or  portions  thereof  marked  with  this  legend  must  also  reproduce  the  markings. 


Appendix  2-14 


Monterey  Technologies,  Inc. 


Contract  No.  DASW01-00-M-4084 


SBIR  Phase  1  Final  Report 


While  a  detailed  investigation  of  the  integration  potential  of  any  of  these  products  was  not  possible,  ServiceSoft  2001  does  not  explicitly  provide 
links  to  other  types  of  reasoning.  The  requirements  for  ATS  to  integrate  several  reasoning  models  may  preclude  its  use  as  the  exclusive  tool  for 
implementing  training  expertise  in  ATS. 

MicroSaint 

MicroSaint  is  a  discrete-event  modeling  and  simulation  tool  It  provides  simple  mechanisms  for  constructing  event-driven  simulations  with 
stochastic  perturbations.  Simulations  are  constructed  from  task  networks  in  which  each  task  is  given  some  behavior.  The  behavior  can 
accomplish  some  business  function  or  set  some  variables 

Users  can  define  variables,  queues,  and  functions  that  are  manipulated  or  used  by  the  tasks  Tasks  are  assigned  mean  and  standard  deviation 
times  that  allow  the  simulation  to  randomly  assign  task  durations.  Several  visual  representations  are  provided  that  provide  displays  of  task 
progress.  Figure  6  presents  a  view  of  the  task  network  for  one  of  the  demonstration  simulations. 


Figure  6:  MicroSaint  Modeling  and  Simulation  Interjace 


In  addition  to  the  task  view  above,  the  action  view  provides  an  animation  for  simulations  that  must  consider  space  issues  in  the  simulation. 
Dynamic  task  queues  and  variable  displays  provide  for  a  lively  simulation  interface 

The  system  is  primarily  used  to  demonstrate  and  test  workflow.  Objects,  tasks,  variables,  functions,  and  queues  combine  to  demonstrate 
bottlenecks  and  underutilizations.  Simply  modifying  the  deviation  of  the  task  duration  can  clearly  identify  brittle  areas  of  an  organizational 
structure. 

For  ATS.  MicroSaint  might  be  used  to  model  the  effect  of  a  student  reacting  too  slowly  to  a  vehicle  motion.  Perhaps  it  could  even  model  the 
effect  of  an  instructor  reacting  to  inadequate  student  performance  Unfortunately,  the  existing  task  structure  could  not  represent  the  flight  model 
of  the  vehicle  and  the  simulation  would  not  provide  adequate  demonstrate  a  true  failure  in  the  control  of  a  hovering  helicopter. 

However,  MicroSaint  recently  added  COM  services  to  its  interface  so  that  it  can  connect  with  other  simulations.  If  ATS  were  to  provide  a  COM- 
based  flight  model  for  use  by  MicroSaint,  then  the  simulations  could  possibly  identify  reaction  time  limits  for  instructional  cues  One  of  the  most 
challenging  aspects  of  providing  helicopter  instruction  is  to  give  the  student  as  much  time  as  possible  to  identify'  and  recover  from  an  unwanted 
flight  regime. 

MicroSaint  currently  sells  for  $8,995. 


MIDAS 

The  U.S.  Army,  NASA,  and  Sterling  Software  Inc.  have  developed  MIDAS  to  aid  in  the  design  of  advanced  aircraft  cockpits.  MIDAS,  the  Man- 
machine  Integration  Design  and  Analysis  System,  combines  graphical  equipment  prototyping,  dynamic  simulations,  and  human  performance 
modeling  to  aid  in  the  design  of  crew  stations  and  their  associated  operating  procedures 
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A  system  of  cooperating  agents,  MIDAS  integrates  editors  and  analysis  tools  written  in  C,  C++,  and  LISP  An  interactive  mode  supports  layout 
of  crew  stations,  assessments  of  visibility  and  legibility,  examination  of  anthropometric  characteristics,  and  analyses  of  cockpit  topology  and 
configuration.  A  simulation  mode  provides  facilities  to  estimate  a  human  operators  employment  of  cockpit  equipment  to  accomplish  mission 
procedures  in  an  integrated  fashion.  Figure  7  presents  a  user's  view  of  MIDAS. 


Figure  1:  User  View  of  MIDAS 

Execution  of  the  simulation  mode  results  in  activity  traces,  task  load  timelines,  information  requirements,  and  mission  performance  measures.  A 
graphics  system  provides  simulation  visualization  and  a  results  analysis  system  supports  examination  of  simulation  data.  An  interface  that  allows 
users  to  construct  target  domain  models  is  also  provided.  Figure  8  presents  an  example  domain  model  object  and  relation  diagram. 


Figure  8.  MIDAS  Domain  Model  Objects  and  Relationships 


Unlike  MicroSaint,  MIDAS  has  the  potential  of  integrating  vehicle  performance  characteristics  into  the  model  of  the  simulation.  Investigations 
into  how  long  an  instructor  could  wait  before  intervening  with  verbal  cues  or  control  augmentation  could  be  carried  out  across  an  array  of 
different  student  models.  The  determination  and  representation  of  this  type  of  temporal  reasoning  within  ATS  presents  a  challenging  task. 
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Instructors  probably  do  not  have  fixed  times  that  they  give  students  to  figure  out  control  problems.  As  a  result,  empirical  observation  and 
simulation  may  be  the  only  available  acquisition  methods. 

The  Easy  Reasoner 

The  Easy  Reasoner  (TER)  suite  of  products,  available  from  The  Haley  Enterprise,  Inc  ,  provides  a  unique  integration  of  several  knowledge-based 
processing  paradigms.  While  most  of  the  reviewed  products  focus  on  a  single  technology,  TER  provides  an  integration  path  between  a  rule-based 
inference  engine  and  a  Case  Based  Reasoning  (CBR)  toolset  in  a  package  that  provides  several  common  interface  options  Figure  9  presents  the 
integration  options  available  with  the  Eclipse  inference  mechanism  that  is  bundled  with  the  ER  product  suite. 


% 


lnEnglnee  428ft  ^  8K*  *♦ 


> 

°°  °v  ^  y 

ct*" 


Figure  9:  Rule-based  Inference  Engines  from  The  Haley  Enterprise 


The  rule-based  inference  mechanism  in  Eclipse  is  based  on  the  Rete  algorithm  Rete  was  proven  to  be  the  most  efficient  production  rule 
execution  process  in  the  late  70's.  Treat  was  subsequently  proven  to  be  faster  for  parallel  execution,  but  the  difference  was  slight.  Derived  from 
the  original  work  done  on  OPS5,  ART,  and  CLIPS,  Eclipse  provides  backward  and  forward  chaining,  truth  maintenance,  taxonomic 
representations,  and  inheritance.  Rete++  provides  a  C++  code  generating  encapsulation  of  Eclipse. 

TER  also  provides  a  rule  induction  capability  for  synthesizing  decision  trees  from  relational  databases  called  ClassIE.  It  automatically  handles 
missing  data  during  induction  and  retrieval  using  probabilities.  Interactive  selection  of  columns  for  induction  and  pruning  of  the  resulting 
decision  trees  provide  users  with  the  ability  to  make  "human  sense"  of  the  induced  tree.  Figure  10  shows  the  relationship  of  the  several  products 
with  TER. 


figure  10:  CBRete  +  +  and  The  Easy  Reasoner  from  Haley 


Case-Based  Reasoning  (CBR)  functions  are  provided  in  another  product  CBRete++  As  you  can  see  in  Figure  10,  this  tool  encapsulates  the 
Rete++  and  TER  products  providing  a  unified  environment  in  which  to  field  solutions  that  integrate  CBR  and  rule-based  reasoning.  Using  direct 
access  to  the  induced  decision  tree  used  for  the  indexed  retrieval  of  cases,  applications  can  implement  interfaces  to  allow  end-users  to  retrieve 
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classified  cases.  The  API  that  provides  access  to  the  tree  also  allows  the  application  to  construct  a  case  programmatically  or  by  form  filling  and 
traverse  the  decision  tree  based  on  the  newly  constructed  case 

Finally,  Haley  offers  yet  another  integrated  product  to  support  problem  diagnosis.  Case-based  Problem  Resolution  (CPR)  employs  case-based 
retrieval  with  rule-based  diagnosis  to  lead  users  to  problem  resolutions.  Case-based  retrieval  alone  does  not  assist  in  identifying  a  problem 
solution  unless  the  problem  case  "exactly"  matches  a  stored  case.  Instead,  decision  procedures  are  assembled  using  independent  rules  that 
identify:  positive  or  negative  evidence  regarding  a  symptom,  the  logical  role  of  some  information,  or  the  evidentiary  role  of  some  information. 

CBRete++  is  priced  at  $4000  and  the  CPR  toolkit  is  $1999.  Runtime  licenses  are  required  for  the  distribution  of  applications.  For  implementing 
ATS,  this  package  provides  the  most  complete  coverage  of  knowledge  representations  and  access  to  underlying  knowledge  structures  of  all  the 
reviewed  technologies.  The  only  shortcoming  is  the  ability  to  import  and  export  standard  knowledge  structure  formats  like  CGIF  or  KIF. 
However,  with  access  to  the  internal  rule  definition  features  of  Rete++,  the  construction  of  an  interchange  mechanism  should  be  simple. 


ATS  Solution  Architecture 

This  section  attempts  to  combine  existing  technology  and  available  products  into  a  plan  for  developing  the  Adaptive  Training  System  (ATS). 

The  helicopter  hover  training  problem  is  first  characterized  and  then  knowledge  structures  organized  to  provide  a  map  to  the  solution  of  building 
the  ATS.  Finally,  suggestions  are  offered  with  respect  to  managing  the  development  and  maintenance  of  the  ATS. 

Hover  Training  Characterization 

The  problem  of  training  a  student  to  hover  a  helicopter  is  analogous  to  the  problem  of  treating  an  illness.  If  you  consider  the  student  pilot  lack  of 
skill  in  performing  a  task  as  an  illness,  then  the  role  of  the  instructor  is  the  same  as  a  doctor  For  both  serious  illnesses  and  aircraft  manipulation, 
an  inability  to  recognize  and  respond  to  situations  can  present  similar  outcomes  The  following  section  presents  the  medical  analogy.  As  you 
read  the  section,  reflect  on  how  each  of  the  concepts  presented  would  operate  in  the  helicopter  training  environment. 

The  Medical  Analogy 

Most  medical  professionals  routinely  perform  a  series  of  tasks  that  attempt  to  improve  the  quality  of  health  in  the  patient.  The  routine  begins  by 
collecting  symptoms  that  describe  the  chief  complaint  of  the  patient.  For  unconscious  patients,  collecting  symptoms  is  guided  by  a  standard 
protocol  that  assumes  the  worse  case  until  proven  otherwise. 

Next,  the  clinician  attempts  to  diagnose  the  cause  of  the  problem  by  correlating  symptoms  and  patient  history  with  known  disease  conditions 
Once  a  plausible  diagnosis  is  reached,  tests  may  be  ordered  to  confirm  the  diagnosis  If  no  plausible  diagnosis  is  reached,  tests  may  be  ordered  to 
begin  the  systematic  process  of  "ruling  out"  the  most  probable  disease  conditions. 

Armed  with  either  a  confirmed  or  suspected  diagnosis,  the  clinician  again  correlates  symptoms  and  patient  history  together  with  the  diagnosis  in 
an  attempt  to  identify  an  appropriate  treatment  for  the  illness.  In  some  cases,  the  illness  itself  is  not  treatable  and  only  the  symptoms  can  be 
addressed.  Standard  treatment  recommendation  guidelines  are  often  available  to  clinicians  for  addressing  specific  symptoms  and  disease 
categories. 

During  treatment  administration,  the  clinician  collects  metrics  that  indicate  the  treatment  effectiveness.  Treatment  side  effects  and  allergies  can 
produce  a  worse  illness  than  the  one  being  treated,  so  a  close  monitoring  of  the  outcome  of  treatment  regime  is  required.  If  the  patient  cannot 
tolerate  the  treatment,  then  it  is  discontinued  and  the  search  for  an  alternative  treatment  begins.  If  the  patient  tolerates  the  treatment  and  shows 
minor  improvements,  an  increase  in  the  treatment  frequency  or  magnitude  may  be  warranted. 

The  clinician  continues  this  potentially  endless  cycle  of  activities  between  diagnostic  evaluation,  treatment  administration,  and  outcomes 
measurement  until  the  patient  is  healed,  killed,  or  simply  doesn't  show  up  any  more.  Since  clinicians  generally  treat  more  than  one  patient, 
outcome  measures  can  be  obtained  on  aggregate  patient  populations.  These  aggregate  outcomes  measures  can  then  be  used  to  estimate  the 
overall  effectiveness  of  a  clinician's  diagnosis-treatment  protocol. 

Finally,  the  medical  associations  collect  statistics  on  clinician  performance  and  identify  areas  in  which  successful  protocols  need  to  be  made 
available  to  clinicians  in  order  to  increase  outcome  success  rates.  The  form  of  these  clinical  guidelines  is  usually  a  paper  protocol,  but  many 
protocols  are  now  being  fielded  as  interactive  knowledge-based  systems. 
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Training  as  Treatment 

With  respect  to  flying  a  helicopter,  the  inability  to  maintain  a  stable  hover  can  be  thought  of  as  a  serious  illness  that  needs  treatment.  In  this  case, 
the  instructor  serves  as  the  clinician  did  in  the  medical  analogy.  Instead  of  a  disease  that  has  a  physiological  basis,  this  disease  is  characterized  by 
a  lack  of  understanding  or  skill.  The  instructors'  treatments  take  the  form  of  practice  sessions  to  refine  perceptual-motor  skills  and  explanations 
to  implant  or  repair  a  student's  conceptual  model  regarding  required  tasks. 

While  a  student  may  identify  a  symptom,  often  the  student  doesn't  even  know  their  sick.  The  instructor  continually  monitors  student  behaviors 
looking  for  symptoms  that  indicate  some  underlying  problem.  A  wandering  rotor  RPM,  airspeed,  and/or  altitude,  are  all  good  symptomatic 
indicators  of  an  underlying  deficiency  in  flying  skills.  A  student  that  is  inattentive  to  the  above  indicators  may  have  a  successful  flight  (walk 
away  from  landing),  but  is  clearly  in  need  of  treatment. 

In  the  same  way  that  clinicians  employ  patient  histories,  instructors  establish  expectations  of  performance  by  looking  at  the  students'  flight  record. 
If  the  instructor  has  flown  with  the  student  before,  then  more  detailed  information  may  be  available  than  can  be  found  in  the  "official''  flight 
record.  Typically  a  logbook  identifies  maneuvers  that  were  executed  (or  attempted),  but  they  do  not  provide  performance  details  regarding  the 
quality  of  the  maneuvers.  The  instructors  experience  with  the  student  may  indicate  a  tendency  to  over  react  or  over  control. 

The  instructor  combines  the  student's  history  with  a  current  skill  assessment  to  select  the  part  of  the  "disease"  process  to  attack  next.  In  the  same 
way  that  clinicians  with  limited  experience  use  treatment  guidelines,  instructors  employ  training  curricula  when  there  is  no  clear  diagnosis  of  the 
next  illness  to  address.  A  training  curriculum  is  really  a  sequence  of  treatment  recommendations  to  be  used  on  all  students  Some  instructors 
simply  push  students  through  these  treatment  recommendations  without  deviation. 

Expert  instructors,  however,  are  able  to  identify  specific  deficiencies  in  student  performance  and  customize  a  training  curriculum  that  is  more 
efficient  than  the  "one  size  fits  all"  training  plan.  The  instructor  is  not  just  attempting  to  minimize  the  time  spent  acquiring  the  skill,  but  is  also 
trying  to  increase  the  safety  of  the  training  process  by  minimizing  the  time  spent  in  a  skill  deficient  state.  Whether  customized  or  canned,  the 
instructor  proceeds  by  administering  treatment  in  the  form  of  a  training  session. 

In  ATS,  training  sessions  are  offered  in  the  form  of  either  a  simulated  flight  or  a  classroom  session.  Simulated  flight  training  sessions  place  the 
student  in  the  1FT  with  a  specific  set  of  training  goals.  Tasks  are  focused  on  vehicle  maneuvering  and  control  manipulation.  Classroom  sessions 
place  the  student  in  front  of  a  computer  interface  with  a  focus  on  establishing  conceptual  structures  needed  to  understand  upcoming  flight  training 
tasks.  While  the  student's  specific  interface  to  ATS  is  different  in  the  two  cases,  each  "treatment"  involves  the  same  process  of  diagnosing  the 
students'  problem,  administering  treatment  and  measuring  outcomes 

Once  ATS  has  applied  its  training  instruction  techniques  to  many  students,  a  history  of  training  efficiency  for  the  ATS  will  be  available.  This 
time  outcome  measures  are  not  indicating  student  pilot  performance,  but  establish  the  quality  of  training  offered  by  the  ATS  instruction.  In  the 
same  way  that  outcome  measures  offer  comments  on  the  effectiveness  of  training  sessions  in  preparing  the  student,  these  ATS  outcomes  indicate 
the  relative  effectiveness  of  the  training  curriculum  implemented  by  the  ATS. 

As  changes  to  the  instructional  curriculum  employed  by  the  ATS  are  made,  relative  outcomes  measures  provide  evidence  regarding  the  efficacy 
of  the  change.  In  this  way,  the  ATS  employs  the  same  mechanisms  for  evaluating  training  recommendations  and  training  curriculums.  Since 
training  curriculums  will  take  the  form  of  a  knowledge-based  system,  the  ATS  will  have  the  ability  to  measure  the  effects  of  modifying  its  own 
reasoning  processes. 

ATS  Components 

The  Easy  Reasoner,  CBRete++,  and  CPR  products  are  proposed  for  the  development  of  the  ATS  system.  These  products  not  only  offer  the 
knowledge  representations  needed  by  ATS,  but  the  components  are  integrated  with  several  object-oriented  programming  API's.  Additionally,  the 
products  are  supported  by  a  company  headed  by  Paul  Haley,  one  of  the  leaders  in  Artificial  Intelligence  since  the  late  70's.  Product  support  and 
specialized  consulting  is  available  from  the  experienced  members  of  The  Haley  Enterprise. 

Visual  C++  on  Windows  2000/NT  is  proposed  as  the  development  platform  and  will  be  used  to  integrate  reasoning  components.  It  will  also  be 
used  to  develop  a  Knowledge  Base  (KB)  Editor.  The  KB  Editor  tool  provides  support  for  Conceptual  Graphs  (CG)  to  facilitate  knowledge 
acquisition  and  analysis.  Visual  representations  for  the  graphs  will  be  manipulated  by  the  KB  editor  and  translated  to  the  Knowledge  Interchange 
Format  (KIF)  for  use  in  ATS.  Conversely,  the  KB  editor  will  read  from  and  permit  the  browsing  of  ATS  knowledge  structures. 

Database  repositories  will  be  maintained  in  a  relational  database.  The  CBRete++  components  utilize  standard  SQL  access,  so  a  standard  Access 
database  will  be  employed  initially.  As  run-time  issues  appear,  alternative  large-scale  databases  will  be  considered  to  replace  Access.  In 
addition,  the  CG  editor  will  provide  the  ability  to  output  CG  graphs  in  a  node  network  form  to  an  Excel  spreadsheet.  This  format  will  allow 
KNOT  to  be  used  to  analyze  the  relations  found  in  both  the  case-base  and  the  concepts  used  to  classify  cases.  Figure  1 1  presents  a  schematic  of 
the  ATS  component  architecture. 
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Figure  II:  A  TS  Component  Architecture 


Voice  Recognition 

The  primary  purpose  for  the  voice  recognition  component  is  to  provide  ATS  with  a  means  of  confirming  the  student’s  understanding.  As  task 
understanding  is  called  into  question,  ATS  can  ask  questions  using  verbal  cueing  and  obtain  responses  directly  from  the  student.  Task 
understanding  can  be  called  into  question  during  either  flight  or  classroom  training  sessions.  The  ATS,  like  an  instructor,  will  have  the  potential 
of  confirming  a  task  understanding  deficiency  and  provide  the  needed  information  without  interrupting  the  current  flight  task. 

The  Voice  Recognition  module  will  be  implemented  in  C++  using  DragonDictate  and  Dragon  Xtools.  DragonDictate  provides  complete  speaker 
independent  voice  control  of  the  entire  Windows  environment.  Dragon  Xtools  provides  a  C++  API  to  provide  control,  configuration,  and 
monitoring  of  voice  recognition  activities  in  DragonDictate.  The  Student  Monitor  component  of  ATS  will  use  Xtools  to  limit  the  expected 
responses  in  order  to  increase  recognition  accuracy 

Student  Monitor 

This  component  is  responsible  for  obtaining  all  required  student  data.  The  primary  source  of  data  obtained  from  the  student  comes  from  the  1FT. 
The  Student  Monitor  (SM)  component  provides  an  Application  Programming  Interface  (API)  that  will  be  used  by  IF T  to  send  the  data.  The  SM 
API  will  be  implemented  using  TCP/IP  sockets  over  a  standard  network  connection  to  the  1  FT 

The  SM  will  be  implemented  in  C++  and  provide  data  preparation  for  attributes  used  to  describe  student  cases.  It  is  anticipated  that  collections  ol 
various  data  post  processing  functions  will  be  implemented  to  support  data  reduction  from  raw  sources  For  example,  a  Fast  Fourier  Transform 
(FFT)  function  will  provide  spectral  analysis  of  the  cyclic  control  input  in  order  to  support  recognizing  when  the  student  is  "over  controlling". 

The  same  types  of  data  and  analysis  will  be  provided  for  establishing  attributes  of  the  helicopter  as  well  This  will  permit  the  Flight  Handling 
Qualities  of  the  helicopter  flight  model  in  IFT  to  be  characterized  along  with  the  student  In  some  simulators,  poor  control  of  the  vehicle  in 
certain  conditions  can  be  the  fault  of  the  flight  model  and  not  the  student  this  analysis  potential  not  only  allows  ATS  to  attribute  the  failure  to  the 
appropriate  component  (student  or  flight  model),  but  also  it  can  identify  areas  where  negative  transfer  of  training  may  be  an  issue  as  the  student 
moves  on  to  real  vehicles. 
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Configuration  and  control  of  the  SM  will  be  provided  by  the  Student  Assessment  component,  including  starting  and  stopping  the  SM  process. 
Depending  on  the  type  of  training  task  being  assessed,  different  types  of  post  processing  of  student  data  may  be  required.  The  SM  will  receive 
instructions  on  what  type  of  data  is  required  and  provide  that  data  to  the  Student  Assessment  component  using  The  Intelligent  Memory  module  of 
CBRete++ 

Student  Assessment 

The  primary  goal  of  this  component  is  to  provide  a  continuous  assessment  of  a  student’s  performance.  It  dynamically  configures  the  Student 
Monitor  component  to  provide  requested  student  metrics  and  then  employs  concept  recognition  knowledge  bases  to  establish  the  student's  skill 
state  and  task  understanding.  The  Student  Assessment  (SA)  component,  like  the  SM  component,  is  configured  specifically  for  the  conduct  of  a 
particular  training  session. 

For  example,  consider  the  "Maintain  Lateral  Position"  task  described  in  Table  1 .  The  acceptable  performance  limits  for  this  task  may  be  driven 
several  factors:  wind  velocity  and  variation,  standard  qualification  standards,  task  duration,  other  required  tasks,  and  the  student’s  past 
performance.  If  there  are  no  other  required  tasks,  then  the  SA  does  not  have  to  be  concerned  with  the  longitudinal  vehicle  position  or  forward 
and  back  cyclic  control  movements.  In  addition,  collective  and  pedal  controls  inputs  are  not  of  concern  to  SA  in  this  case.  SA  selects  only  those 
concepts  relevant  to  the  task(s)  at  hand  for  monitoring  and  evaluation 

The  SA  will  be  implemented  in  C++  using  the  CBRete++  and  CPR  components.  The  CBRete++  components  will  be  used  to  establish  the 
student's  training  progress  as  the  current  problem  case.  The  CPR  components  will  be  used  to  decide  on  relevant  concepts  and  metrics  for  the 
given  task(s).  As  you  recall,  the  CPR  components  provide  an  environment  for  employing  case-based  reasoning  together  with  classical  diagnostic 
production  rules. 

Representing  knowledge  that  determines  relevant  concepts  in  the  context  of  multiple  tasks  and  varied  student  experience  is  complex.  The  SA 
will  manage  this  complexity  by  representing  specific  relations  between  tasks,  concepts,  metrics,  and  historic  performance  independently  using 
rules  and  cases.  Tasks,  or  combinations  of  tasks,  will  imply  relevant  concepts  Relevant  concepts  and  task  specific  historic  performance  implies 
acceptance  criteria  and  metrics.  The  rule  bases  employed  by  SA  will  implement  instructor  expertise  with  regard  to  assessing  a  student  in  the 
context  of  required  tasks  and  experience. 

The  SA  configuration  information  establishing  the  student  under  test  and  required  tasks  is  provided  by  the  Training  Assessment  component.  The 
Intelligent  Memory  (TIM)  component  of  the  CBRete++  component  facilitates  the  sharing  of  context  information.  In  addition,  the  Training 
Assessment  component  is  responsible  for  the  starting  and  stopping  of  the  SA  process 

Training  Assessment 

The  Training  Assessment  (TA)  component  is  the  main  control  point  for  the  ATS  Runtime  environment.  ATS  users  interact  with  the  TA 
Graphical  User  Interface  (GUI)  in  order  to  control  and  monitor  the  complete  training  environment,  including  the  1FT.  The  TA-GUI  begins  by 
establishing  the  student  identification  and  setting  default  training  goals 

The  primary  goal  of  the  TA  component  is  to  represent  expert  instructor  knowledge  with  regard  to  adapting  a  training  curriculum  to  a  specific 
student.  The  TA  represents  knowledge  needed  to  decide  what  the  next  training  task  should  be  in  the  context  of  a  particular  student's  case  history. 
For  new  students  that  the  system  has  no  history  of,  some  form  of  standard  training  curriculum  might  be  employed  for  the  specific  training  goal 
selected.  For  students  that  have  interacted  with  the  system  before,  the  tasks  would  be  customized  so  as  to  optimize  training  efficiency  for  that 
student. 

The  TA  also  has  responsibility  for  controlling  several  aspects  of  the  IFT.  Flow  control  involves  the  starting  and  stopping  of  the  1FT.  Remote 
flight  control  involves  the  pausing  (and  continuing)  of  the  flight  model  in  addition  to  managing  any  flight  control  augmentation.  Remote  voice 
control  passes  phrases  that  the  TA  needs  spoken  to  the  student  through  the  IFT.  Remote  environmental  control  manages  values  like  wind 
direction  and  speed  used  by  the  IFT  flight  model  and  visibility  used  by  the  IFT  image  generators. 

In  order  to  support  the  required  interfaces  to  the  IFT,  the  TA  provides  an  API  that  will  be  used  in  creating  the  ATS  IFT.  Figure  12  presents  a 
schematic  of  the  ATS  IFT  and  its  relationship  to  the  ATS  Both  the  TA  and  SM  API's  will  be  employed  to  construct  a  fully  encapsulated  version 
of  the  IFT  for  use  by  ATS.  Once  started,  the  ATS  IFT  process  connects  to  the  TA  through  the  TA  API.  Further  connections  to  the  SM  are  made 
as  required  based  on  requested  tasks 
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Figure  12:  Encapsulating  IFF  for  integration  with  ATS 


The  TA  will  be  implemented  in  C++  using  the  CBRete++  and  CPR  components  The  CBRete++  components  will  be  used  to  establish  the 
student's  training  program  using  previous  training  cases  and  induced  decision  trees.  It  will  also  be  used  to  establish  the  current  instruction 
parameters  as  an  instruction  case  in  support  of  post-instruction  evaluation  of  the  ATS  instruction  methods.  The  CPR  components  will  be  used  to 
decide  on  relevant  training  tasks  given  the  student's  performance. 

Initially,  a  student's  history  is  consulted  to  establish  the  required  tasks  for  the  selected  training  goal(s).  The  tasks  are  sent  to  the  Student 
Assessment  (SA)  module  for  use  in  configuring  metrics  and  subsequently  establishing  the  attributes  required  for  monitoring  by  the  SM.  The 
knowledge  employed  by  the  TA  in  determining  the  training  tasks  is  a  combination  of  ease  based  reasoning  and  diagnosis,  as  in  the  SA 
component. 

For  example,  a  new  student  may  be  given  an  initial  task  of  lifting  the  helicopter  to  a  5-foot  hover  and  maintaining  lateral  and  horizontal  position 
within  5  feet  in  a  no  wind  condition.  In  this  case  all  controls  are  active  and  not  augmented  If  the  student  performs  this  task  within  specified 
criteria,  then  the  system  moves  on  to  the  next  task  in  the  curriculum  If  the  student  cannot  respond  within  the  specified  criteria,  the  SA  indicates 
the  types  of  errors  using  the  student  assessment  attributes.  If  the  student  initially  lost  control  bv  over-controlling  the  lateral  cyclic  input,  then  the 
case  base  and/or  diagnostic/treatment  knowledge  bases  may  suggest  reducing  the  number  of  controls  to  lateral  only  and  provide  some  minor 
control  augmentation 

The  process  of  determining  whether  Case-Based  Reasoning  (CBR)  or  classical  diagnostic  rule  based  processing  is  used  to  represent  specific 
instructional  knowledge  is  the  focus  of  knowledge  acquisition  and  engineering  activities  In  the  beginning  of  system  development  when  a  case 
base  does  not  exist,  interviews  with  instructors  will  focus  on  identifying  the  taxonomy  of  student  training  progression  and  errors.  This  knowledge 
might  be  represented  initially  in  the  form  of  a  classical  diagnostic  production  system  As  students  are  encountered,  the  case  base  grows  and  the 
performance  of  the  current  instructional  knowledge  base  established 

During  offline  analysis  of  the  student  case  base  and  the  instructional  knowledge  base,  a  decision  tree  is  induced  from  the  case  base.  The 
differences  between  this  decision  tree  and  the  existing  diagnostic  rule  base  are  carefully  analyzed  to  identify  potentially  missing  concepts  in  the 
rule  base.  If  differences  exist  but  cannot  be  readily  identified  at  this  time,  the  case  could  be  added  to  the  case  base  and  solution  responses 
indicated  by  an  expert.  In  the  future  similar  cases  will  be  adapted  to  this  solution 
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If  cases  are  simply  remembered,  you  might  wonder  why  it  would  ever  be  desirable  to  do  the  work  of  "re-representing"  the  knowledge  in  the  case 
base  using  a  diagnostic  rule  base.  The  issue  is  one  of  understanding  and  explanation.  A  Case-Based  Reasoning  system  doesn't  really  know  why 
it  is  making  a  decision.  The  system  simply  correlates  cases  and  adapts  previously  determined  solutions  The  process  of  migrating  concepts 
discovered  in  the  case  base  to  diagnostic  rule  bases  is  the  primary  mechanism  for  producing  explanations  of  the  ATS  training  curricula.  The 
Knowledge  Base  Editor  provides  the  analysis  and  modification  of  knowledge  bases  employed  by  the  SA  and  TA  components. 

Knowledge  Base  Editor 

This  component  is  the  main  control  point  for  the  ATS  Development  environment  The  KB  Editor  (KBE)  provides  for  human  access  to  all  stored 
case  base  and  rule  base  information  manipulated  by  SA  and  TA  The  primary  aspects  of  knowledge  base  management  supported  by  the  KBE  are 
the  following: 


1.  Concept  analysis  of  the  student  case  base  and  induced  decision  trees, 

2.  Identification  of  student  cases  for  use  in  decision  making, 

3.  Concept  analysis  of  the  instructor  case  base  and  induced  decision  trees, 

4.  Identification  of  instructor  cases  for  use  in  decision  making, 

5.  Editing  and  management  of  all  diagnostic  rule  bases, 

6.  Presentation  of  knowledge  in  the  form  of  Conceptual  Graphs, 

7.  Import  and  Export  of  knowledge  in  KIF  and  CGIF 

8.  Export  of  appropriate  knowledge  in  spreadsheet  form. 

Aspects  1  and  2  are  identical  to  3  and  4  except  that  they  deal  with  the  student  and  instructor,  respectively.  The  student  case  base  is  analyzed  to 
characterize  the  types  and/or  progression  of  errors  that  students  make  during  training.  The  instructor  case  base  is  analyzed  to  characterize  the 
types  and/or  progression  of  instruction  techniques  that  succeed  in  achieving  training  goals. 

Aspects  5  and  6  provide  typical  rule  editing  and  rule  base  management  functions.  The  graphical  presentation  of  rule  networks,  case  specific 
execution  traces,  and  other  knowledge  base  debugging  functions  are  incrementally  developed  as  needed.  The  display  of  rules  and  rule  networks 
in  the  form  of  conceptual  graphs  should  aid  in  the  need  for  the  real-time  review  of  existing  knowledge  bases  during  knowledge  acquisition 
interviews. 

Aspect  7  and  8  provide  for  the  interoperability  of  knowledge  structures  in  other  tools.  Network  analysis  using  KNOT  is  supported  with  the 
spreadsheet  export.  Other  information  analysis  tools  that  accept  KIF  or  CGIF  can  be  used  in  the  generation  or  analysis  of  ATS  knowledge. 
CBRete++  will  maintain  the  knowledge  base  in  a  relational  database.  Initially,  an  inexpensive  database  will  be  used  until  it  becomes  a 
bottleneck.  Depending  on  the  efficiencies  of  the  access  mechanisms  and  underlying  hardware,  other  databases  may  be  substituted  later  in  the 
program. 

System  Production  and  Maintenance 

ATS  is  a  complicated  knowledge  based  system.  The  successful  development  of  ATS  requires  the  employment  of  several  tools  to  help  manage 
the  complexity  of  the  development  project  itself.  The  CommonKADS  methodology  is  proposed  to  organize  the  construction  of  the  components 
within  ATS.  RationalRose,  an  object-oriented  modeling  tool,  is  proposed  to  represent  the  models  required  by  the  CommonKADS  methodology. 
All  aspects  of  the  ATS  project  will  be  modeled  to  account  for  all  activities  and  provide  explicit  representations  for  dependencies  between 
components  and  requirements. 

Microsoft  Project  2000  is  proposed  to  manage  the  schedule  of  development  activities  during  the  ATS  development  cycle.  Many  students  and 
instructors  will  be  needed  for  knowledge  acquisition  and  system  testing.  Cognitive  psychologists  and  knowledge  engineers  must  collaborate  on 
conceptual  discovery  and  case-based  analysis  activities.  It  is  anticipated  that  a  major  key  to  the  success  of  bringing  the  program  in  on  time  and 
budget  will  be  careful  management  of  the  limited  personnel  resources 
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Conclusions 


While  the  goals  of  the  ATS  program  are  ambitious,  the  technical  risks  associated  with  the  ATS  development  program  are  minimal.  A  mature, 
integrated,  and  supported  product  technology  is  available  from  a  leading  Artificial  Intelligence  (AI)  tool  vendor.  The  ability  to  represent 
knowledge  structures  in  multiple  forms,  for  use  in  several  decision-making  paradigms,  supports  incremental  refinement.  This  incremental 
refinement  capability  encourages  the  use  of  ATS  during  its  own  development  process. 

Finally,  while  the  design  described  tailored  to  helicopter  hover  training  using  the  IFT,  the  structural  elements  and  philosophy  should  support 
adaptive  training  in  many  student/instructor  applications. 
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